|
Patch to reload mime types each type yaws starts
|
View:
New views
5 Messages
—
Rating Filter:
Alert me
|
 |
Patch to reload mime types each type yaws starts
by Koushik Narayanan-3
::
Rate this Message:
Hi,
I had a requirement wherein changes to /etc/mime.types come into effect without requiring rebuilding yaws. So I patched yaws to use the ets table created while parsing mime.types for returning mime types and to rebuild the ets each time yaws starts. The patch is on Yaws-1.77 tree. Let me know if there are better ways of doing this. Also the idea of compiling in mime type mappings is for throughput ? Such behaviour causes deployment issues. Regards, Koushik Narayanan [yaws-mime.patch] diff -uNr yaws-1.77.orig/src/Makefile yaws-1.77.work/src/Makefile --- yaws-1.77.orig/src/Makefile 2009-10-29 17:40:28.000000000 +0530 +++ yaws-1.77.work/src/Makefile 2009-10-29 19:39:48.000000000 +0530 @@ -78,13 +78,13 @@ else rm charset.def 2> /dev/null; touch charset.def; fi mime_types.erl: mime.types mime_type_c.erl charset.def - $(ERL) -noshell -pa ../ebin -s mime_type_c compile + ##$(ERL) -noshell -pa ../ebin -s mime_type_c compile debug: $(MAKE) TYPE=debug clean: - rm -f $(EBIN_FILES) yaws_generated.erl charset.def mime_types.erl + rm -f $(EBIN_FILES) yaws_generated.erl charset.def install: regen all docsinstall $(INSTALL) -d $(DESTDIR)$(PREFIX)/lib/yaws/examples/ebin diff -uNr yaws-1.77.orig/src/mime_type_c.erl yaws-1.77.work/src/mime_type_c.erl --- yaws-1.77.orig/src/mime_type_c.erl 2009-10-29 17:40:28.000000000 +0530 +++ yaws-1.77.work/src/mime_type_c.erl 2009-10-29 19:40:18.000000000 +0530 @@ -21,28 +21,31 @@ c() -> - {ok, F} = file:open("mime.types", [read]), - io:format("Compiling mime.types ... > mime_types.erl ~n", []), - {ok, B} = file:read_file("charset.def"), - case string:tokens(binary_to_list(B)," \r\n\t" ++ [0, 12]) of - [] -> - put(charset, []); - [CharSet0] -> - CharSet = string:strip(CharSet0, both, 10), - put(charset, ";charset=" ++ CharSet); - _ -> - error_logger:format("Ignoring bad charset.def\n", []), - put(charset, []) - end, - T = ets:new(aa, [set, public]), + {ok, F} = file:open("/etc/mime.types", [read]), + %%io:format("Compiling mime.types ... > mime_types.erl ~n", []), + io:format("Generating mime_types table from /etc/mime.types~n", []), +% {ok, B} = file:read_file("charset.def"), +% case string:tokens(binary_to_list(B)," \r\n\t" ++ [0, 12]) of +% [] -> +% put(charset, []); +% [CharSet0] -> +% CharSet = string:strip(CharSet0, both, 10), +% put(charset, ";charset=" ++ CharSet); +% _ -> +% error_logger:format("Ignoring bad charset.def\n", []), +% put(charset, []) +% end, + T = ets:new(mime_types, [set, public,named_table]), c(F, T, io:get_line(F, '')). c(F, T, eof) -> file:close(F), - gen(T); + T; + %gen(T); c(F, T, {error, terminated}) -> file:close(F), - gen(T); + ok; + %gen(T); c(F, T, Line) -> case Line of [$#|_] -> @@ -50,7 +53,7 @@ [$\s|_ ] -> ignore; L -> - case string:tokens(L, [$\s, $\t]) of + case string:tokens(L, [$\s, $\t,$\n]) of [] -> ignore; [_] -> diff -uNr yaws-1.77.orig/src/mime_types.erl yaws-1.77.work/src/mime_types.erl --- yaws-1.77.orig/src/mime_types.erl 1970-01-01 05:30:00.000000000 +0530 +++ yaws-1.77.work/src/mime_types.erl 2009-10-30 12:49:13.000000000 +0530 @@ -0,0 +1,19 @@ +-module(mime_types). + +-export([t/1,revt/1]). + + +t("yaws") -> {yaws,"text/html"}; +t("php") -> {php,"text/html"}; +t("cgi") -> {cgi,"text/html"}; +t("PHP") -> {php,"text/html"}; +t("CGI") -> {cgi,"text/html"}; + + +t(String) -> + case ets:lookup(mime_types,String) of + [{String,MimeType}] -> {regular,MimeType}; + _ -> {regular,"text/plain"} + end. + +revt(String) -> t(lists:reverse(String)). diff -uNr yaws-1.77.orig/src/yaws_sup.erl yaws-1.77.work/src/yaws_sup.erl --- yaws-1.77.orig/src/yaws_sup.erl 2009-10-29 17:40:28.000000000 +0530 +++ yaws-1.77.work/src/yaws_sup.erl 2009-10-29 20:27:00.000000000 +0530 @@ -24,6 +24,7 @@ %%% API %%%---------------------------------------------------------------------- start_link() -> + mime_type_c:c(), supervisor:start_link({local, ?MODULE}, ?MODULE, []). %%%---------------------------------------------------------------------- ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Erlyaws-list mailing list Erlyaws-list@... https://lists.sourceforge.net/lists/listinfo/erlyaws-list |
|
|
Re: Patch to reload mime types each type yaws starts
by Steve Vinoski-2
::
Rate this Message:
On Sat, Oct 31, 2009 at 7:49 AM, Koushik Narayanan <koushik.list@...> wrote:
Hi, Technically, you need to rebuild only the mime types module. So I patched yaws to use the ets table created while parsing mime.types for returning mime types and to rebuild the ets I guess that depends on how you use it. One, if you update the mime types and rebuild the mime types module, all you need to do is redeploy its beam file, not all of yaws. Second, if you start yaws in a manner that allows you to remsh or rpc into it, you can leave it running and just hot-reload the new mime types module on the fly without restarting yaws at all. This reload logic could easily be written in a small shell script, for example.
Also, you might want to consider updating to 1.85, as I did a huge update on the MIME types recently, adding quite a few in the process. Perhaps with this newer version, the need to reload mime types will be greatly reduced or even eliminated for you.
--steve ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Erlyaws-list mailing list Erlyaws-list@... https://lists.sourceforge.net/lists/listinfo/erlyaws-list |
|
|
Re: Patch to reload mime types each type yaws starts
by Koushik Narayanan-3
::
Rate this Message:
Hi,
On Sat, Oct 31, 2009 at 7:28 PM, Steve Vinoski <vinoski@...> wrote:
Yes. The scenario is I have a binary package of yaws given to a system administrator wants to add/remove MIME types.
Ok. But that would mean having an erl file generated at the deployment machine and compiled there. Though the erlang installations have both the compiler and the runtime installed, generally it wouldn't be required to have a compiler at a deployment machine.
I am stuck at 1.77 because, I am having problems in using the OTP http module to fetch data from yaws asynchronously in yaws versions greater than 1.77. And also I feel its not required to have a comprehensive list of all possible mime types at all deployments. Depending on the application specifying what is required would be better imho. Koushik Narayanan ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Erlyaws-list mailing list Erlyaws-list@... https://lists.sourceforge.net/lists/listinfo/erlyaws-list |
|
|
Re: Patch to reload mime types each type yaws starts
by Steve Vinoski-2
::
Rate this Message:
On Sat, Oct 31, 2009 at 11:12 AM, Koushik Narayanan <koushik.list@...> wrote:
Hi, Why does it have to be generated at the deployment host? A beam file is a beam file; it can be generated anywhere and run anywhere. You don't really need a separate compiler, BTW, given that the compile module is always available by default in Erlang/OTP unless you take it out. If you're changing mime.types, you can generate a new one from an erlang shell by calling mime_type_c:compile(), assuming the yaws ebin directory is in the erlang code path and your working directory is where mime.types resides. This will generate a new mime_types.erl source module. You can then compile that either using the c(mime_types) erl shell command, which will also load it for you, or by calling compile:file("mime_types") and then using the code module to load the object code. Either way, doing this from a shell script is quite easy. Also, this can all be done via a remsh to a running yaws instance, in which case generating and compiling the new mime_types module will load it right into your running web server.
I guess I disagree. MIME types are registered with the IANA and are thus globally standardized and widely known. There's really no reason to subset them unless you're running in an embedded environment with memory limitations so severe that it takes too much space to hold MIME type definitions you'll never use.
IMO the overall problem here is that by using just a subset of the MIME types, you then have to compensate for that at each deployment host. If you just deployed the right set to begin with, it wouldn't be an issue.
When you say you have problems fetching data asynchronously via the OTP http module, I assume you're talking about the client side, so how can that be a yaws problem? Assuming the responses from yaws conform to RFC 2616, which I think is safe to assume otherwise we'd be getting all kinds of bug reports, the problem would seem to be in the client. Have you considered using the ibrowse client instead? There are a few known bugs in the httpc module.
My overall advice is to 1) determine why your client has an issue, and fix that so you can move off 1.77 to a more recent yaws, and 2) to deploy a much more complete set of MIME types such as that found in 1.85, such that your sysadmins do not need to generate anything. If you need help fixing the client issue, please send the details of the problems you're seeing to this list, and we'll try to help as much as we can.
--steve ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Erlyaws-list mailing list Erlyaws-list@... https://lists.sourceforge.net/lists/listinfo/erlyaws-list |
|
|
Re: Patch to reload mime types each type yaws starts
by Koushik Narayanan-3
::
Rate this Message:
Hi,
On Sat, Oct 31, 2009 at 10:00 PM, Steve Vinoski <vinoski@...> wrote:
Agreed. But adding lines to a text file would certainly be easier.
IANA allows usage of x- (extended) MIME types which are not required to be registered with it. Though its usage is discouraged in RFC2045, they are in widespread use. Also there are vendor specific mime types which are required when interfacing with specific clients. So it is very difficult to build up a set of all required MIME types.
Yes. Its a bug in the httpc module. Since development was done on 1.77 it was not noticed. Will be changing the http client soon after which will upgrade to the recent yaws release.
Yes I will move to 1.85 soon. I will be shifting the http client to ibrowse or the http client from ETC. ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Erlyaws-list mailing list Erlyaws-list@... https://lists.sourceforge.net/lists/listinfo/erlyaws-list |
| Free embeddable forum powered by Nabble | Forum Help |
