Sending attachments

[Richard Stallman]

    I don't know about you, but I'm a lot more afraid of CC-mode's code than
    of Gnus's.  If you agree with that and following your reasoning, we
    should have kept the old c-mode.el.

CC mode stays in its own area of functionality -- it has no tendency
to spawn replacements for other parts of Emacs.  I am confident that,
if Alan thinks some new non-C-specific feature is needed for CC mode,
he will discuss the right way to develop it as a new module.

Thus, while its complexity is a problem (and will cause difficulty if
Alan disappears), it is a contained problem.

[Richard Stallman]

    > There are two reasons.  First, its primary source is maintained
    > outside of Emacs.  Second, the people who work on it do not hesitate
    > to add dependencies between one part and another, with results that
    > one quickly sees on studying files from Gnus.

    I'm sure they'll appreciate help in reducing dependencies between modules.

I am sure they would, but that doesn't excuse creating all those
dependencies.

    Actually, they originally did make a specific effort to make
    message-mode independent from Gnus so that it can replace mail-mode.

Maybe they wanted it to replace Mail mode, but they didn't ask me if I
wanted to replace it.  They should have talked with me first, and then
I would have said, "Please extend Mail mode, and please talk with me
about what you want to do."

[Stephen J. Turnbull]

Frank Schmitt writes:

 > Is XEmacs compatibility really worth the trouble? How many XEmacs users
 > are still out there?

First, XEmacs compatibility is not usually that great a burden.
Certainly, when you're trying to support bleeding edge features (viz.
preview-latex), you run into problems.  But mostly the work is already
done, and with a few exceptions where XEmacs led Emacs by 5 to 10
years in introducing an API and Emacs deliberately varied, we do
(eventually) sync by adding the feature with the Emacs-style APIs.

Second, there are still plenty of XEmacs and SXEmacs users to justify
*keeping* existing features.  Several GNU/Linux distros have tried to
drop XEmacs packages or deemphasize them in one way or another, and
gotten *strong* pushback from their users.  XEmacs also still has a
strong following in two large companies that I know of, I believe
because they consider it much easier to maintain local extensions to
XEmacs.  They don't even try with Emacs.  I have to think that there
are a lot of such users out there, who probably don't participate much
in free software channels, they just use it.

There's also the question of *contributors* vs. users.  XEmacs (and
even more so, SXEmacs) contributors have been quite loyal, due to the
large investment required to work on internals of any Emacsen.  So
making XEmacs less usable would quite likely result in a loss of
developers to the entire community rather than a flow of developers to
Emacs proper.

Anyway, the question is if merging Gnus code into Emacs would result
in a purge of XEmacs compatibility code.  Sometimes Richard uses
expressions that suggest that might happen, but AFAICT his opinion on
the matter is that he thinks mixing XEmacs compatibility code with
application implementation is unclean, and he advocates *separating*
them out.  Of course if XEmacs compatibility modules are not going to
be maintained, he prefers them to be dropped, and I'm sure he'd prefer
that the effort be devoted to Emacs proper -- but after all, that's
the contributor's choice.

[Jason Rumney-4]

Stephen J. Turnbull wrote:
> Anyway, the question is if merging Gnus code into Emacs would result
> in a purge of XEmacs compatibility code.
To clarify, the question is about merging SOME Gnus code into Emacs. I
would see it working the same way as the code that has been merged from
Gnus already, where the Gnus folk continue to offer their own packages
for XEmacs and older versions of Emacs that include the files that were
merged into Emacs plus any compatibility code that has been modularized
and doesn't need to be in the Emacs repository.

[Stephen J. Turnbull]

Richard Stallman writes:

 > Someone asked me why I see this as a problem, and I explained why.
 > You may not see it as a problem, but the point is that I do.

Sure.  And my point is that I don't understand why you see it as a
problem.  I like to understand these things, and I'm at a loss to come
up with a sane rationalization.  So I feel very very stuck.  That
would be OK if it were just me, but I don't think I'm the only one who
feels that way.  In fact I think some of those who are feeling stuck
are important contributors to Emacs, and to free software.

 >     It seems to me that both Gnus and Emacs would benefit from you
 >     pow-wowing with Miles Bader and anybody else who works on Gnus-Emacs
 >     integration, and make the points about refactoring and maintainability
 >     to them.
 >
 > I would like to see progress in this direction, and I am willing to do
 > a certain amount of work to help it.  However, at present I have
 > nothing to specific suggest, because I know very little about the code
 > of Gnus.

What you've posted about rfc2047.el already goes way beyond
"nothing". :-)

 > Its complexity has discouraged me from looking at it except in
 > cases of need.

Unfortunately, that complexity is mostly a reflection of the problem
domain.  It is *not* merely a reflection of the number of features of
Gnus; reducing the complexity of your requirements will not reduce the
complexity of a RFC conforming implementation proportionately.  For
example, you personally may have no need to send non-ASCII characters
in headers.  Nevertheless, the MUA you use must check for them,
because they might be present in an automatically generated header
(eg, a reply).  Of course you can simplify somewhat by refusing to
implement RFC 2047, and simply have `mail-send' signal an error,
leaving it to the user to clean up the header.  But the check itself
is complexity, and goes beyond what Mail mode has ever done.  And it
gets worse; RFC 2047 by itself is insufficient.  Even if you've
implemented RFC 2047, you need to check whether parameter values of
MIME headers contain non-ASCII, because RFC 2047 itself specifically
prohibits use of RFC 2047 encoded-words in parameters.  Now, you also
need RFC 2231.  Mail is rife with these corner cases.

This is not mere conformance for the sake of conformance.  To give one
example, Mailman for many years had a bug where a non-ASCII character
in a header could cause a whole installation and all of its lists to
become unusable until a sysadmin shunts the offending message out of
the queue.

 > If Gnus developers are interested in doing work to make these files
 > modular, I'm interested in looking for candidates.

Thank you!

[Ted Zlatanov]

On Thu, 16 Jul 2009 13:19:07 +0800 Jason Rumney <jasonr@...> wrote:

JR> Stephen J. Turnbull wrote:
>> Anyway, the question is if merging Gnus code into Emacs would result
>> in a purge of XEmacs compatibility code.
JR> To clarify, the question is about merging SOME Gnus code into Emacs. I
JR> would see it working the same way as the code that has been merged
JR> from Gnus already, where the Gnus folk continue to offer their own
JR> packages for XEmacs and older versions of Emacs that include the files
JR> that were merged into Emacs plus any compatibility code that has been
JR> modularized and doesn't need to be in the Emacs repository.

On Wed, 15 Jul 2009 14:27:46 -0400 Stefan Monnier <monnier@...> wrote:

>> Pulling code out of Gnus has the downside that it then needs to be
>> inside Emacs, but XEmacs may not have it.

SM> Don't worry about this part: the changes (supposedly) needed don't
SM> intend to remove any code, just better modularize it (e.g. so that
SM> message-mode can be self-standing without importing (transitively)
SM> gnus-*.el packages).

As long as we're talking about changes local to the Gnus project (not
pulling things out to Emacs itself) I think this is reasonable.  If,
however, we plan to pull libraries out to Emacs, as I think some of the
comments are suggesting, we need to ensure it will not break XEmacs
compatibility.  Thanks for the clarification.

Ted

[Ted Zlatanov]

On Wed, 15 Jul 2009 21:10:58 -0400 Richard Stallman <rms@...> wrote:

RS> I don't know how many other parts of Gnus are comparable to rfc2047.el
RS> in that they are useful general facilities and ought to be made
RS> separate and modular.  If Gnus developers are interested in doing work
RS> to make these files modular, I'm interested in looking for candidates.

If we establish a list of specific things, as Reiner's list for example,
it will be easier to work on it.  As I mentioned, there aren't that many
Gnus developers and refactoring code is a thankless task that tends to
take tons of time.  You should expect this to be a slow process.

Ted

[Richard Stallman]

    Pulling code out of Gnus has the downside that it then needs to be
    inside Emacs, but XEmacs may not have it.  XEmacs compatibility has (as
    long as I can remember) been a Gnus goal.

1. XEmacs can have these files if its developers want.

2. There could be a Gnus-XEmacs compatibility package
released alongside Gnus.

[Ted Zlatanov]

On Thu, 16 Jul 2009 12:17:52 -0400 Richard Stallman <rms@...> wrote:

RS>     Pulling code out of Gnus has the downside that it then needs to be
RS>     inside Emacs, but XEmacs may not have it.  XEmacs compatibility has (as
RS>     long as I can remember) been a Gnus goal.

RS> 1. XEmacs can have these files if its developers want.

RS> 2. There could be a Gnus-XEmacs compatibility package
RS> released alongside Gnus.

I think Reiner Steib and Stephen Turnbull, among others, need to discuss
this with you and the Emacs maintainers.  I'll go along with whatever
decision the group makes.

Ted

[Stephen J. Turnbull]

Ted Zlatanov writes:

 > RS> 1. XEmacs can have these files if its developers want.
 >
 > RS> 2. There could be a Gnus-XEmacs compatibility package
 > RS> released alongside Gnus.
 >
 > I think Reiner Steib and Stephen Turnbull, among others, need to discuss
 > this with you and the Emacs maintainers.  I'll go along with whatever
 > decision the group makes.

Mike Kupfer should be in the mix, other than that I don't have a
particular opinion on what to do.

[Richard Stallman]

    If we establish a list of specific things, as Reiner's list for example,
    it will be easier to work on it.  As I mentioned, there aren't that many
    Gnus developers and refactoring code is a thankless task that tends to
    take tons of time.  You should expect this to be a slow process.

I understand that.  My point is, first do no harm.

Since it is hard work to remove these dependencies,
please avoid making more of them.  When you want to add
a new file, ask yourself whether its function is inherently
part of Gnus, or whether it is something general (like rfc2047.el
for example) that you simply want for Gnus.  And if it is the latter,
please write it in a way that doesn't depend on Gnus, and propose
it for inclusion in Emacs.

[Ted Zlatanov]

On Fri, 17 Jul 2009 09:30:30 -0400 Richard Stallman <rms@...> wrote:

RS>     If we establish a list of specific things, as Reiner's list for example,
RS>     it will be easier to work on it.  As I mentioned, there aren't that many
RS>     Gnus developers and refactoring code is a thankless task that tends to
RS>     take tons of time.  You should expect this to be a slow process.

RS> I understand that.  My point is, first do no harm.

RS> Since it is hard work to remove these dependencies,
RS> please avoid making more of them.  When you want to add
RS> a new file, ask yourself whether its function is inherently
RS> part of Gnus, or whether it is something general (like rfc2047.el
RS> for example) that you simply want for Gnus.  And if it is the latter,
RS> please write it in a way that doesn't depend on Gnus, and propose
RS> it for inclusion in Emacs.

I'll try to watch for this in new submissions and in my own work.

Ted

[Richard Stallman]

rfc2047.el depends on some substantial code in mm-util.el:
mm-charset-to-coding-system and mm-find-mime-charset-region.  I cannot
understand that code, and I am not sure what it is trying to do.  I
understand the doc strings, but they explain only in vague terms.

Can you separate out that functionality into a file by itself, which
does not depend on the rest of Gnus, and clean it up and/or add
comments so that it is clear?

[Richard Stallman]

    RS> part of Gnus, or whether it is something general (like rfc2047.el
    RS> for example) that you simply want for Gnus.  And if it is the latter,
    RS> please write it in a way that doesn't depend on Gnus, and propose
    RS> it for inclusion in Emacs.

    I'll try to watch for this in new submissions and in my own work.

Thank you.  Things will be a lot better that way.

[Reiner Steib]

On Sat, Jul 18 2009, Richard Stallman wrote:

> rfc2047.el depends on some substantial code in mm-util.el:
> mm-charset-to-coding-system and mm-find-mime-charset-region.  I cannot
> understand that code, and I am not sure what it is trying to do.  I
> understand the doc strings, but they explain only in vague terms.

Please elaborate what is not clear...

,----[ <f1> f mm-find-mime-charset-region RET ]
| mm-find-mime-charset-region is a compiled Lisp function in `mm-util.el'.
| (mm-find-mime-charset-region B E &optional HACK-CHARSETS)
|
| Return the MIME charsets needed to encode the region between B and E.
| nil means ASCII, a single-element list represents an appropriate MIME
| charset, and a longer list means no appropriate charset.
`----

,----[ <f1> f mm-charset-to-coding-system RET ]
| mm-charset-to-coding-system is a compiled Lisp function in `mm-util.el'.
| (mm-charset-to-coding-system CHARSET &optional LBT ALLOW-OVERRIDE SILENT)
|
| Return coding-system corresponding to CHARSET.
| CHARSET is a symbol naming a MIME charset.  [...]
`----

MIME charsets are not the same as (Emacs) coding systems.  This
function does the mapping and also handles overrides (many MUAs
specify wrong charsets), invalid charsets, etc.

> Can you separate out that functionality into a file by itself, which
> does not depend on the rest of Gnus, and clean it up and/or add
> comments so that it is clear?

rfc2047.el implements (one of) the MIME standards.  mm-utils.el
contains utility functions for MIME.  I.e. they are closely related.
I'm not sure it is worth to separate out parts of it.

Bye, Reiner.
--
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

[Mike Kupfer-3]

>>>>> "ST" == Stephen J Turnbull <stephen@...> writes:

ST> Ted Zlatanov writes:

RS> 1. XEmacs can have these files if its developers want.

RS> 2. There could be a Gnus-XEmacs compatibility package released
RS> alongside Gnus.

>> I think Reiner Steib and Stephen Turnbull, among others, need to
>> discuss this with you and the Emacs maintainers.  I'll go along
>> with whatever decision the group makes.

ST> Mike Kupfer should be in the mix, other than that I don't have a
ST> particular opinion on what to do.

Hi, folks.  I'm the Gnus maintainer for XEmacs.  Stephen T. pointed me
at this discussion.

The XEmacs-specific code in Gnus appears in one of two ways (that I'm
aware of):

1. XEmacs-specific files (gnus-xmas.el, messagexmas.el)
2. (featurep 'xemacs) checks in otherwise common code.

I just did a quick pass through the Gnus 5.10.10 sources to see how
often (featurep 'xemacs) appears.  I get 45 hits for message.el plus
mm*.el (171 hits for Gnus as a whole).  A lot of those 45 hits are in
menu generation code, leaving 18 or so places that will like require
more serious work for refactoring.

gnus-xmas.el and messagexmas.el are distributed with the Gnus releases
from gnus.org.  I'd like to see those files remain with the Gnus
sources, as separating them would increase the risk of breakage as
changes are made to the common code.

mike

[Richard Stallman]

    > rfc2047.el depends on some substantial code in mm-util.el:
    > mm-charset-to-coding-system and mm-find-mime-charset-region.  I cannot
    > understand that code, and I am not sure what it is trying to do.  I
    > understand the doc strings, but they explain only in vague terms.

    Please elaborate what is not clear...

If only a piece of it were unclear, that would be a reasonable question.
However, the problem is that I can't even begin to understand most
of those functions.  The necessary info is not present.

    MIME charsets are not the same as (Emacs) coding systems.  This
    function does the mapping and also handles overrides (many MUAs
    specify wrong charsets), invalid charsets, etc.

That is just the beginning of the necessary explanation for
understanding what this code does.  To state the purpose in general
terms like this is not enough.

To make the code clear means explaining the specifics of what it does.
Each group of a few lines needs comments to explain why those lines
are there.  Lots of background information needs to be provided.

    rfc2047.el implements (one of) the MIME standards.  mm-utils.el
    contains utility functions for MIME.  I.e. they are closely related.

Only part of mm-utils.el is closely related to rfc2047.el.  That part
is what I am talking about here.  It consists of the two functions
mm-find-mime-charset-region and mm-charset-to-coding-system, and their
subroutines and data.

    I'm not sure it is worth to separate out parts of it.

I don't think you and I are talking about the same "it".  The things
you think I want to separate, actually I want to keep together.

I am going to move rfc20457.el outside Gnus to make it a regular part
of Emacs.

What I want to do is keep the two functions
mm-find-mime-charset-region and mm-charset-to-coding-system (and their
subroutines and data) together with rfc2047.el, making them too a
regular part of Emacs.  This requires separating them from the rest of
mm-utils.el which will remain inside Gnus.  It also requires making
them clean and understandable.

I am asking Gnus developers to help me by do this by splitting mm-utils
and cleaning up this part.

If I have to do this on my own, then since I can't understand all the
code of these functions, I will have to delete parts of the code until
I can understand what remains.  Some features will be lost, but at
least it will be clear and maintainable.  Whatever features I had to
discard could be re-added later if someone can write them cleanly.

However, if you help, maybe we can achieve a better outcome in which
we get clean code that implements all the existing features of
mm-charset-to-coding-system and mm-find-mime-charset-region, and thus
have no inconvenience for anyone.

[Richard Stallman]

     > Its complexity has discouraged me from looking at it except in
     > cases of need.

    Unfortunately, that complexity is mostly a reflection of the problem
    domain.

I think you are mistaken.  I think most of the complexity has nothing
to do with this.

[Stefan Monnier]

> If only a piece of it were unclear, that would be a reasonable question.
> However, the problem is that I can't even begin to understand most
> of those functions.  The necessary info is not present.

I'm not familar with those functions, but they don't look particularly
unclear to me.  So I don't know what "necessary info" you're looking for.

>     rfc2047.el implements (one of) the MIME standards.  mm-utils.el
>     contains utility functions for MIME.  I.e. they are closely related.

> Only part of mm-utils.el is closely related to rfc2047.el.  That part
> is what I am talking about here.  It consists of the two functions
> mm-find-mime-charset-region and mm-charset-to-coding-system, and their
> subroutines and data.

They're used in other places as well (basically to encode message
bodies), so they don't really belong to rfc2047.el.

> I am going to move rfc20457.el outside Gnus to make it a regular part
> of Emacs.

It's been part of Emacs since Emacs-21.  And I don't think you can
prevent the Gnus maintainers from distributing rfc2047.el along
with Gnus.

> What I want to do is keep the two functions
> mm-find-mime-charset-region and mm-charset-to-coding-system (and their
> subroutines and data) together with rfc2047.el, making them too
> a regular part of Emacs.  This requires separating them from the rest
> of mm-utils.el which will remain inside Gnus.  It also requires making
> them clean and understandable.

Rather than focus on code-ownership, I'd rather we focus on this latter
part: "clean and understandable".

> I am asking Gnus developers to help me by do this by splitting mm-utils
> and cleaning up this part.

Maybe one way to look at it is to split mm-utils.el into a part that
deals with compatibility between different Emacsen
(e.g. mm-string-to-multibyte) and the other that provides actual
functionality (e.g. mm-find-mime-charset-region).

I'm still wondering why someone would want to do that since it seems
pretty far from the goal of improving the user's experience.
IOW, another way to look at this problem would be: what changes would it
take to make Rmail use message-mode for composition?  I'm sure this will
take less time and make more people happier.


        Stefan

[Eli Zaretskii]

> From: Richard Stallman <rms@...>
> Date: Sun, 19 Jul 2009 00:36:49 -0400
> Cc: tzz@..., ding@..., emacs-devel@...
>
> What I want to do is keep the two functions
> mm-find-mime-charset-region and mm-charset-to-coding-system (and their
> subroutines and data) together with rfc2047.el, making them too a
> regular part of Emacs.

Do we really need these two functions?  I think we have the necessary
infrastructure in Emacs:

  . find-charset-region

  . find-coding-systems-region

  . find-coding-systems-for-charsets

  . (coding-system-get FOO :mime-charset)

  . coding-system-list

  . sort-coding-systems

If these are not enough to write a much cleaner, leaner versions of
those two mm-utils functions, please tell what else is missing.