[scala] Scala plugin BETA

[sean.mcdirmid]

Hi Amir,

We went through this argument 25 years ago :) People just don't
like structured editors, and there are good reasons for that:
they interrupt a programmer's train of thought, they are slower.
See all the work on the Cornell Program Synthesizer (or something
like that), and the results of that work.

Its an interesting area of research though.

BTW, I fixed your semi-colon inference disabled in
embedded XML bug. Which sort of implies I've been
messing with XML....

引言 Amir Michail <amichail@...>:

> On 9/19/07, Amir Michail <amichail@...> wrote:
> > On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:
> > > We can definitely parse the whole block, and if people type only
> > > well-formed XML in each edit, we have no problem. Of course,
> > > thats called a structure editor, and they are incredibly unusable.
> > >
> > > How do we handle intermediate edits? E.g., someone begins to
> > > type some XML...the XML parser doesn't have much error recovery
> > > in it, it will only determine that the block is invalid when it
> hits
> > > EOF. It can't tell us where the incomplete XML block ends.
> > >
> > > Thats the big issue, how do we handle people editing XML?
> > >
> >
> > People generally enter XML in a structured way, so take advantage of
> that.
> >
>
> BTW, I believe that such issues would be non-existent with an
> IDE-based approach to programming where one is always working with
> syntactically correct code.
>
> See:  http://en.wikipedia.org/wiki/Intentional_programming
>
> Amir
>
> > Amir
> >
> > > Sean
> > >
> > > $B0z8@(B Amir Michail <amichail@...>:
> > > > On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:
> > >
> > > > What's wrong with parsing the entire XML block?  Is that hard to
> do
> > > > with the current plugin architecture?
> > >
> > >
> >
>

[Alex Boisvert-3]

On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:

Definitely! These are the most interesting bugs: the build compiler
says yes but the presentation compiler chokes on them. Its
usually because I'm not adapting right (T required on T, for some
reason the T symbol is being duplicated and not reused), or
because my incremental compiler algorithm does something
that the batch compiler algorithm doesn't. They are based on
the same code, but there are a lot of hacks to make type checking
incremental.

Curious, did you build first before you opened up the files in the
editor? This sounds like the IDE file wasn't found or something,
but it could be that your code does something I didn't anticipate.

I created a new project, copied the file over to the new project.   When I opened the file for the first time, everything looked good, but when I started making changes the presentation compiler errors appeared.

I created an issue at:  http://lampsvn.epfl.ch/trac/scala/ticket/118

alex

[Amir Michail]

On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:
> Hi Amir,
>
> We went through this argument 25 years ago :) People just don't
> like structured editors, and there are good reasons for that:
> they interrupt a programmer's train of thought, they are slower.
> See all the work on the Cornell Program Synthesizer (or something
> like that), and the results of that work.
>
> Its an interesting area of research though.

From http://web.cs.wpi.edu/~gpollice/cs509-s04/Readings/simonyi95death.pdf:

"The syntax directed editors have come the closest to approximating
the IP editing problem. In the syntax directed paradigm, the syntax
rules are immediately applied to the user's input, with two important
results: first the input of illegal constructions is prevented at the
outset; and second, as a result of parsing, the system has knowledge
of what has been typed and this knowledge can be used in turn to help
the user with prompts, automatic formatting, defaults and so on.
Despite these advantages, syntax directed editing has never been a
commercial success. One can only surmise the reasons for the user's
dislike of such systems, but one good guess is that a successful
system should be modeless and intentional. Modelesssness means that
work on one section of the code can be always suspended and the focus
transferred to some other portion. Intentional here means that the
editor commands should directly relate to the intentions rather than
the representation of the intentions."

One advantage of this approach is that you can have ambiguous
notation.  For example, you can use mathematical notation as is even
though it is highly ambiguous -- people will generally know what you
mean from context.

BTW, it amazes me that scientists are still using TeX/LaTeX even
though there's a much better alternative, namely the syntax-directed
WYSIWYG TeXmacs.

Amir

>
> BTW, I fixed your semi-colon inference disabled in
> embedded XML bug. Which sort of implies I've been
> messing with XML....
>
> 引言 Amir Michail <amichail@...>:
>
> > On 9/19/07, Amir Michail <amichail@...> wrote:
> > > On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:
> > > > We can definitely parse the whole block, and if people type only
> > > > well-formed XML in each edit, we have no problem. Of course,
> > > > thats called a structure editor, and they are incredibly unusable.
> > > >
> > > > How do we handle intermediate edits? E.g., someone begins to
> > > > type some XML...the XML parser doesn't have much error recovery
> > > > in it, it will only determine that the block is invalid when it
> > hits
> > > > EOF. It can't tell us where the incomplete XML block ends.
> > > >
> > > > Thats the big issue, how do we handle people editing XML?
> > > >
> > >
> > > People generally enter XML in a structured way, so take advantage of
> > that.
> > >
> >
> > BTW, I believe that such issues would be non-existent with an
> > IDE-based approach to programming where one is always working with
> > syntactically correct code.
> >
> > See:  http://en.wikipedia.org/wiki/Intentional_programming
> >
> > Amir
> >
> > > Amir
> > >
> > > > Sean
> > > >
> > > >  $B0z8@ (B Amir Michail <amichail@...>:
> > > > > On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:
> > > >
> > > > > What's wrong with parsing the entire XML block?  Is that hard to
> > do
> > > > > with the current plugin architecture?
> > > >
> > > >
> > >
> >
>
>

[Vlad Dumitrescu-2]

Hi!

On 9/19/07, sean.mcdirmid@... <sean.mcdirmid@...> wrote:

> We can definitely parse the whole block, and if people type only
> well-formed XML in each edit, we have no problem. Of course,
> thats called a structure editor, and they are incredibly unusable.
>
> How do we handle intermediate edits? E.g., someone begins to
> type some XML...the XML parser doesn't have much error recovery
> in it, it will only determine that the block is invalid when it hits
> EOF. It can't tell us where the incomplete XML block ends.
>
> Thats the big issue, how do we handle people editing XML?

If I may propose a solution I am using in the Erlang plugin, it's not
perfect but works well enough:

If the file being edited is in a stable state (i.e. can be lexed
successfuly) and the use starts to type at position X in the file, we
can assume that the lexical tokens that were originally after position
X will remain the same after the edit is done (i.e. the user will
enter a number of complete tokens). This means that lexing can be done
just in the region that is being edited, so you can know where the XML
is supposed to end.

this wasn't the best explanation I ever gave, but I hope it's understandable.

best regards,
Vlad

[sean.mcdirmid]

This is a decent solution and I actually use something like it
in the plugin: if you edit the end of a line, an invisible ";" will
be inserted; e.g., in

foo.
bar

if you just wrote the '.', the it won't consume bar as the
selector of the expression, instead, it will give a syntax
error. This way, the type information @ bar isn't destroyed
by the overzealous parser. The implicit semi colon
goes away once you stop editing the line.

I thought about doing this for XML, but the logistics
are complicated. I've hacked in an alternative solution
for now that I'll announce in another email.

Sean

引言 Vlad Dumitrescu <vladdu55@...>:

> If I may propose a solution I am using in the Erlang plugin, it's not
> perfect but works well enough:
>
> If the file being edited is in a stable state (i.e. can be lexed
> successfuly) and the use starts to type at position X in the file, we
> can assume that the lexical tokens that were originally after position
> X will remain the same after the edit is done (i.e. the user will
> enter a number of complete tokens). This means that lexing can be done
> just in the region that is being edited, so you can know where the XML
> is supposed to end.
>
> this wasn't the best explanation I ever gave, but I hope it's
> understandable.
>
> best regards,
> Vlad
>

[Judson, Ross]

A few more random thoughts...

When a language like Scala embeds XML, the lexer/parser makes a "flip"
into another syntax. We pretty need to be able to switch back and forth
to embed Scala back into the XML:

class Something(toast: String) {
  val hello = <hello>{toast}</hello>
  def apply() { Console.println(hello) }
}

Scala uses the curly brace convention to switch back and forth. What if
we were to generalize the notion of embedded languages, and we somehow
wanted the Scala compiler to be able to handle more than just XML...like
if we wanted to extend in the direction of LINQ.

Approach 1: How much syntactic distance would have to be covered to
allow for XML literals to actually _be_ valid Scala syntax. That's
probably not possible, but it's interesting to think about.

Approach 2: Providing an alternate syntax that needs to manipulate
Scala's AST is fairly complex. If we restrict inline parsing to the
generation of literal values (with embedded Scala expressions) or simple
functions returning values, there's no need have knowledge of the AST.

As the parser combinators are part of the standard library, it's
conceivable that scalac could manifest a declared parser inline and pass
it a sequence of tokens of either DSL text or embedded Scala fragment
results, which would result in a function returning some kind of
well-typed value (this is somewhat analogous to the XML processing done
in nsc). So in the case of XML the general shape would be:

trait LiteralToken;
case class ScalaValue(value: () => Any) extends LiteralToken;
case class DSLText(text: String) extends LiteralToken;

trait LiteralParser[T] {
  def apply(): T
  val processor: String
  val markerRegex: (Option[String], Option[String])
  val scalaEscapeRegex = ("\\{", "\\}")
}

type XMLBuilder = () => scala.xml.Elem

class XMLParser extends Parsers with LiteralParser[XMLBuilder] {
  val processor = "XML"
  val markerRegex = (Some("<[a-zA-Z]"), None)
  type Elem = LiteralToken;
  ...  
}

When executed at runtime, the parser would create a function that, when
called, would produce an XML element. The scala compiler would be
responsible for parsing the "escapes" to Scala down to functions
suitable for the ScalaValue tokens. Bringing a literal parser's "marker"
signature in scope as an implicit would "enable" literal parsing of that
syntax to take place. Some kind of literal start indicator similar to an
XML processing instruction could be used to direct parsing to a
particular processor, if inline literal start marks are inadequate or
inappropriate:

def someXML(name: String) = '?XML' <some>xml {name}</some> '?XML'

def findIt(resource: String, number: Int) = '?xquery'
doc({resource})//invoice[@number={number} '?xquery'

There are significant parallels between parser combinators and the
stream-based unification used when doing logic programming in a
functional language (see The Reasoned Schemer, or Martin's scala-prolog
example). A logic parser would have goal functions as its token type,
and produce a stream of substitution/binding lists (logical primitives
such as condi, alli etc would be combinators). Declarative inline
processing based on the standard combinators could conceivably cover
some of the logic domain, shifting the burden into libraries that both
aid the compiler and augment the runtime.

RJ

[sean.mcdirmid]

Strange, I don't know whats going on. Could you tell us what the  
warning is? Annotations should be OK, I've seen them used in the code  
I've tested, but I'm not sure about Java 1.5 annotations.

Maybe we should remove the target option from the plugin. I've  
realized I only have everything setup to work with 1.5 right now, so  
even if you select 1.4, things shouldn't work.

Sean

On Sep 19, 2007, at 2:52 PM, n8han wrote:

>
> (Sorry if this dups, but after a day my Gmane message is still not  
> showing.)
>
> Does this beta support targeting JDK 1.5? I created a new project  
> with the
> beta, and set target JDK 1.5 in the Scala panel of the project  
> options, but
> I'm still getting warnings when I use an annotation.
>
> Otherwise, looks great!
>
> Nathan
> --
> View this message in context: http://www.nabble.com/-scala--Scala- 
> plugin-BETA-tf4473177.html#a12776781
> Sent from the Scala mailing list archive at Nabble.com.
>

[n8han]

Yes, I should say I'm talking about Java / JDK 1.5 annotations. And the set of preferences I'm referring to, they might be a relic. It's the "Scala Compiler Properties" page that also exists in the old plugin, in project preferences. The default for that form, in the old and new plugins, is target jvm-1.4. New values are retained, but setting 1.5 does not stop the warning. I don't know of anywhere else that the jvm target can be configured in the eclipse plugin. My eclipse java compiler compliance settings are for "5.0".

The warning that appears in the problems view is:
Java annotation will not be emitted in classfile unless you use the '-target:jvm-1.5' option

This is the same warning generated if I try to compile with scalac and the default jvm target. You can trigger it with any built-in Java annotation, like java.lang.Override. There's a bug for this in the old plugin:
http://scala-webapps.epfl.ch/bugtracking/bugs/displayItem.do?id=1025

But contrary to what it says there, my class files do not have the annotation information. At least, at runtime the annotation can not be found by Hibernate. If I recompile externally with the 1.5 target param the annotation is found and everything works. I've tried many times with the previous generation of the plugin, and I've just now tested beta2. If it ever does work it would be wonderful for me, but I understand that Java annotations are not such a high priority for most people. And I know that, if I need it that bad, I should just dive right in and try to fix it, though it's a little daunting since I haven't done any eclipse plugin or scala compiler programming.

Nathan

sean.mcdirmid wrote:

Strange, I don't know whats going on. Could you tell us what the  
warning is? Annotations should be OK, I've seen them used in the code  
I've tested, but I'm not sure about Java 1.5 annotations.

Maybe we should remove the target option from the plugin. I've  
realized I only have everything setup to work with 1.5 right now, so  
even if you select 1.4, things shouldn't work.

Sean

On Sep 19, 2007, at 2:52 PM, n8han wrote:

> Does this beta support targeting JDK 1.5? I created a new project  
> with the
> beta, and set target JDK 1.5 in the Scala panel of the project  
> options, but
> I'm still getting warnings when I use an annotation.
>
> Otherwise, looks great!
>
> Nathan