gputils development

[David Barnett-3]

It's been my experience that gputils has a steep learning curve for
development, probably more so than the average open source project.

I think the reasons are probably that:
 - the source language (C) is fairly low-level
 - the project has changed hands several times and never had a large
development team
 - the scope (translating PIC assembly) is pretty complicated and not very
exciting on the face of it

I've set up a page on the gputils wiki with some background and a place for
suggestions: http://gputils.wiki.sourceforge.net/FosteringInnovation. Please
feel welcome to freely add/modify, but please keep specific feature-related
discussion in the Feature Request tracker or the FeatureWishlist page (for
more ambitious suggestions).

I'd like to throw out a few questions:
 - First, am I making a big deal about nothing? Is gputils' current
development cycle okay, slow pace, dead periods and all?
 - Has anyone started to dig through the source code and make changes, but
given up? What might have helped you continue (e.g. specific questions
answered, other developers working in parallel, IRC bug days...)?
 - For developers who have contributed big patches (Borut, Michael Ballbach,
Andreas Kabel, many more...), do you have any ideas or experience to share?
 - What features added in recent history have been most helpful (new COFF
support, CONFIG directive on PIC18, etc.)?
 - Has anyone switched back to MPASM because of a feature missing from
gputils?

I haven't had much energy lately to work on gputils, but I'd like to do
whatever little things I can (answer questions, touch up docs, work closely
with other devs) to help accelerate development.

David

[Xiaofan Chen]

On Wed, Nov 12, 2008 at 1:24 AM, David Barnett <daviebdawg@...> wrote:
> It's been my experience that gputils has a steep learning curve for
> development, probably more so than the average open source project.

Or maybe the whole open source PIC development is not
so fun. gputils is good. gpsim is ok. sdcc for PIC has a long
way to go. Piklab is not moving. pk2cmd works fine. Still no
good debugger support. gnupic mailing list is quiet.

On the other hand, Microchip is moving very fast with
PIC24/dsPIC/PIC32, USB, TCP/IP, Zigbee, etc. Most
of the tools are provided under Windows. So maybe
many people (including me) are moving to PIC24 and
PIC32 since both offer good gcc based compilers which
work under Linux and Mac OS X.

Example for PIC32:
http://www.paintyourdragon.com/uc/osxpic32/index.html
http://forum.microchip.com/tm.aspx?m=364626

Now there is a new chance for lower end PIC16F with
the announcement of Enhanced PIC16F.
http://www.microchip.com/enhanced
http://forum.microchip.com/tm.aspx?m=378667

This should be a good update to people using
PIC16F. Hopefully gputils/sdcc/gpsim can catch the
opportunity.

Just some random thoughts.

Xiaofan

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[vaclavpe]

> On Wed, Nov 12, 2008 at 1:24 AM, David Barnett <daviebdawg@...> wrote:
> > It's been my experience that gputils has a steep learning curve for
> > development, probably more so than the average open source project.
>

I am working in Linux, programming PICs. But sometimes I need to work in Windows as well. Maybe my comments (as an almost-BFU) would be helpful for somebody.
> so fun. gputils is good.
Agree - I have no problems so far

> gpsim is ok.
Yes, ok. I don't use gpsim too much. Maybe somebody can clarify - can I simulate the code written in C-language compiled with SDCC ? And what about code from Microchip MCC18 ? (I mean stepping through C-lang lines, not ASM lines)

> sdcc for PIC has a long way to go.
I am regularly experimenting with SDCC. A lot of things work. I can't get USB code working compiled with SDCC. I have to use MCC18 for it. Still do not konw why.  And maybe I would like to see some more optimizations to have code more compact. I do not need reentrant functions etc.

> Piklab is not moving.
I never used it

> pk2cmd works fine.
That is true. But I really miss chip autodetection mechanism similar to Windows' PicKit2 programmer. And I do not know how to show revision of the PIC. Maybe nice-to-have: automatically show ID and revision of the target PIC every time pk2cmd is run.

> Still no good debugger support.
That is definitely true. Does ANY debugger for PIC on Linux exist (even some bad debugger) ? Support of PicKit2 debugger will be really cool.

Vaclav

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[David Barnett-3]

On Thu, Nov 13, 2008 at 4:18 AM, Vaclav Peroutka <vaclavpe@...> wrote:

> > gpsim is ok.
> Yes, ok. I don't use gpsim too much. Maybe somebody can clarify - can I
> simulate the code written in C-language compiled with SDCC ?

Last I heard, you can't step through C lines. That's why gpsim always prints
"FIXME: HLL files are not supported at the moment" on the terminal.

BTW Xiaofan, I think gputils could help with some problems that seem like
they're SDCC-only. For instance, a lot of people hesitate to use C because
of the code bloat, and SDCC doesn't do a ton of optimization compared to the
alternatives, but some link-time optimizations could really cut down on the
bloat.

David

[Xiaofan Chen]

On Fri, Nov 14, 2008 at 4:15 AM, David Barnett <daviebdawg@...> wrote:
> BTW Xiaofan, I think gputils could help with some problems that seem like
> they're SDCC-only. For instance, a lot of people hesitate to use C because
> of the code bloat, and SDCC doesn't do a ton of optimization compared to the
> alternatives, but some link-time optimizations could really cut down on the
> bloat.

That is a good idea. But IMHO Microchip has other ideas about
this topic. They are working on LLVM port. In fact, LLVM 2.4
has experimental PIC16 ports.
http://llvm.org/docs/ReleaseNotes.html

Check out LLVM mailing lists and you will know that several
Microchip employees are working on the PIC16 port.

Xiaofan

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[vaclavpe]

>
> That is a good idea. But IMHO Microchip has other ideas about
> this topic. They are working on LLVM port. In fact, LLVM 2.4
> has experimental PIC16 ports.
> http://llvm.org/docs/ReleaseNotes.html
>

Hello Xiaofan,

I had a look into LLVM docu, Getting started etc. There is "1. Read the documentation, 2.Read the documentation". But which documantation ? I could not find what is LLVM good for. And how can it help me with development for PIC16 architecture.

Is it something like simulator ? Or profiler ? Or optimizer ?

I agree with David that SDCC output code is quite fat. But this is because of internal structure of SDCC, intermediate code etc.... Does it mean that with LLVM I can optimize the code generated with SDCC ?

Vaclav

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[David Barnett-3]

On Fri, Nov 14, 2008 at 12:36 AM, Xiaofan Chen <xiaofanc@...> wrote:

> That is a good idea. But IMHO Microchip has other ideas about
> this topic. They are working on LLVM port. In fact, LLVM 2.4
> has experimental PIC16 ports.
> <http://llvm.org/docs/ReleaseNotes.html>

I had noticed that. Link-time optimization is one of the big advantages of
LLVM. I seem to remember some users suggesting that SDCC use LLVM when I
followed their mailing list.

I've been wondering how gputils can take advantage of LLVM. It seems like
the LLVM architecture would be an alternative to most of gputils, minus a
PIC14 backend. But I wonder how much LLVM could help with the limited call
stack and pages/banks on the mid-range PICs. I've always been interested in
offloading runtime work into a smarter toolchain instead of the software
developer.

David

[David Barnett-3]

On Fri, Nov 14, 2008 at 4:04 AM, Vaclav Peroutka <vaclavpe@...> wrote:

> >
> > That is a good idea. But IMHO Microchip has other ideas about
> > this topic. They are working on LLVM port. In fact, LLVM 2.4
> > has experimental PIC16 ports.
> > http://llvm.org/docs/ReleaseNotes.html
> >
>
> Hello Xiaofan,
>
> I had a look into LLVM docu, Getting started etc. There is "1. Read the
> documentation, 2.Read the documentation". But which documantation ? I could
> not find what is LLVM good for. And how can it help me with development for
> PIC16 architecture.

LLVM is a whole framework geared towards compiler developers, and still not
much of a product for end-users. I don't think they have much user
documentation yet, but the project looks very promising.

If you don't know all about compilers, there's usually a front-end which
translates from C, FORTRAN, Ada, C++, etc. into an intermediate
representation, and a back-end which translates the intermediate
representation into assembly or machine language for a particular arch (x86,
ARM, etc.). LLVM is a definition of the intermediate representation and a
lot of tools for dealing with it (optimizing, debugging...). With gcc the IR
is much more hidden, but LLVM standardizes the IR format and makes it easier
to enhance the whole system.

They've added a PIC16 back-end now, so you can use a front-end (like
llvm-gcc, I think) to generate the IR code, and then get PIC16 assembly or
machine code out of that, theoretically. I haven't looked much into it
myself, but that should be the gist of it.

David

[Walter Banks]

David Barnett wrote:

> On Fri, Nov 14, 2008 at 4:04 AM, Vaclav Peroutka <vaclavpe@...> wrote:
>
> > >
> > > That is a good idea. But IMHO Microchip has other ideas about
> > > this topic. They are working on LLVM port. In fact, LLVM 2.4
> > > has experimental PIC16 ports.
> > > http://llvm.org/docs/ReleaseNotes.html
> > >
> >
> > Hello Xiaofan,
> >
> > I had a look into LLVM docu, Getting started etc. There is "1. Read the
> > documentation, 2.Read the documentation". But which documantation ? I could
> > not find what is LLVM good for. And how can it help me with development for
> > PIC16 architecture.
>
> LLVM is a whole framework geared towards compiler developers, and still not
> much of a product for end-users. I don't think they have much user
> documentation yet, but the project looks very promising.
>
> If you don't know all about compilers, there's usually a front-end which
> translates from C, FORTRAN, Ada, C++, etc. into an intermediate
> representation, and a back-end which translates the intermediate
> representation into assembly or machine language for a particular arch (x86,
> ARM, etc.). LLVM is a definition of the intermediate representation and a
> lot of tools for dealing with it (optimizing, debugging...). With gcc the IR
> is much more hidden, but LLVM standardizes the IR format and makes it easier
> to enhance the whole system.
>
> They've added a PIC16 back-end now, so you can use a front-end (like
> llvm-gcc, I think) to generate the IR code, and then get PIC16 assembly or
> machine code out of that, theoretically. I haven't looked much into it
> myself, but that should be the gist of it.
>

There have been several attempts at defining standardized optimizations. All
of them including LLVM work quite well once they are written at optimizing
code for architectures similar to the original target. The problem that most
have not solved is making the optimizations relevant as the target architectures
change. LLVM's approach to standardizing IR doesn't solve this problem.

In the end it will be a test of how well the IR maps on the target. This
mapping efficiency will determine the effectiveness of LLVM on
PIC targets. To illustrate my point look at the subtract instruction on
the Microchip PICs the argument order is reversed from most processors
which in turn affects the data flow analysis need for the orderly arrival of
data to the alu on the processor. It is this type of small thing that makes
LLVM type of optimization approach fail to realize expectations.


w..




---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Ralph Corderoy]

Hi Walter,

> There have been several attempts at defining standardized
> optimizations. All of them including LLVM work quite well once they
> are written at optimizing code for architectures similar to the
> original target. The problem that most have not solved is making the
> optimizations relevant as the target architectures change. LLVM's
> approach to standardizing IR doesn't solve this problem.

No, but it does allow common optimisations for free, and make
experimentation with architecture-specific ones easier.  Other aspects
of LLVM, IIRC, include the advantage of taking sufficient information
from the source all the way through to link time, allowing Link Time
Optimisations, e.g. in theory, discarding an unused global debug flag
and all the pieces of code that check it.

There's lots of interesting papers published on the LLVM site, here's
just a couple.

    http://llvm.org/pubs/2005-05-21-PLDI-PoolAlloc.html
    http://llvm.org/pubs/2005-06-12-MSP-PointerComp.html

They also have their Clang C compiler as an easuer one to experiment
with than gcc.

Cheers,


Ralph.


---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Xiaofan Chen]

On Fri, Nov 14, 2008 at 10:14 PM, David Barnett <daviebdawg@...> wrote:
>
> They've added a PIC16 back-end now, so you can use a front-end (like
> llvm-gcc, I think) to generate the IR code, and then get PIC16 assembly or
> machine code out of that, theoretically. I haven't looked much into it
> myself, but that should be the gist of it.
>

It seems that Microchip employees (seems to be from India design center)
are working on clang front end (http://clang.llvm.org/).

Xiaofan

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Scott Dattalo]

Walter Banks wrote:

> LLVM's approach to standardizing IR doesn't solve this problem.

As a matter of principal, one cannot disagree. (And any of you who are
unfamiliar with Walter should know that he is the most experienced PIC
C-compiler writer on this list... And perhaps the most experienced in the
world! - his opinion probably weighs in the most on this subject.)

In my experience in dabbling with SDCC, I found that the IR (intermediate
representation) is a somewhat inadequate paradigm. Even mostly similar
microcontrollers have different/optimal requirements for the IR. I tried
to mitigate this somewhat by creating a second layer of IR unique to PICs
called pCode (prior to this, SDCC only had an IR called iCode). In
hindsight, I learned this approach is somewhat analogous to gcc's backend
RTL. I started my SDCC efforts about the time gputils linker and object
code approach were becoming. Had I stuck with this project, I would have
extended pCode to the linker. This seems to be the new addition that LLVM
brings to the table.

Somewhat off topic, but I've been thinking about the concept of 'process'
lately. Some people seem to think if you have the write 'process' in place
that the chances of success are increased. But ultimately, success hinges
on execution. Successful execution depends on individuals. And successful
individuals seldom need a process for guidance. The LLVM framework defines
a process for writing compilers. Its success ultimately hinges on those
performing the execution...

Scott


---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Ralph Corderoy]

Hi Scott,

> Walter Banks wrote:
> > LLVM's approach to standardizing IR doesn't solve this problem.
>
> As a matter of principal, one cannot disagree. (And any of you who are
> unfamiliar with Walter should know that he is the most experienced PIC
> C-compiler writer on this list... And perhaps the most experienced in
> the world! - his opinion probably weighs in the most on this subject.)

LLVM is more than has been suggested by concentrating on just its IR and
I wouldn't like to see LLVM dismissed lightly.  It seems Microchip, who
should know a thing or two about the awkwardness of targetting the PIC's
architecture, agree.

> Had I stuck with this project, I would have extended pCode to the
> linker. This seems to be the new addition that LLVM brings to the
> table.

It's one.  LLVM is a large beast, but they've tried to avoid it getting
too complex.  The people behind it are academics and researchers who
have battled with gcc internals in the past and know what they dislike
about it.

> Somewhat off topic, but I've been thinking about the concept of
> 'process' lately. Some people seem to think if you have the write
> 'process' in place that the chances of success are increased. But
> ultimately, success hinges on execution. Successful execution depends
> on individuals. And successful individuals seldom need a process for
> guidance. The LLVM framework defines a process for writing compilers.

No, it provides much more than just a framework;  there's real hard work
done in there for you, work which makes adding to it to experiment with
architecture specific issues easier.  It's more than a lexer, parser,
and IR, more than, e.g. the lcc compiler.
http://www.cs.princeton.edu/software/lcc/

> Its success ultimately hinges on those performing the execution...

The people behind LLVM are top-notch, and Chris Lattner has shown he's
able to lead a project, providing clear direction, without preventing
others from making large contributions.  Apple are now providing
corporate backing.  Cutting edge researchers have been attracted to
using it for their experiments.  Yes, I agree, the results of
Microchip's efforts need to be examined, but I suspect there's more
chance of them steadily improving with LLVM compared with, for example,
if they'd used some gcc fork.

Anyway, there's no need to argue amongst ourselves about this.  I just
don't want to see LLVM dismissed lightly before it's been given a few
iterations to see how it copes with the oddities of PIC compared with
its existing targets.  We'd all like better FLOSS PIC C support;  I
happen to think LLVM might help us get there, and have done for some
years.  I was delighted to see a Microchip email address pop up on the
LLVM list some time ago.

Cheers,


Ralph.


---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Scott Dattalo]

Ralph Corderoy wrote:

> Anyway, there's no need to argue amongst ourselves about this.

True Ralph, one shouldn't look a gift horse in the mouth! Getting one's
head wrapped around gcc's internals is difficult. If the LLVM efforts
allow smart, motivated people to more easily create PIC compilers, then we
should all agree (on this list) that that is a good thing!

Scott



---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Xiaofan Chen]

On Fri, Nov 14, 2008 at 4:15 AM, David Barnett <daviebdawg@...> wrote:
> On Thu, Nov 13, 2008 at 4:18 AM, Vaclav Peroutka <vaclavpe@...> wrote:
>
>> > gpsim is ok.
>> Yes, ok. I don't use gpsim too much. Maybe somebody can clarify - can I
>> simulate the code written in C-language compiled with SDCC ?
>
> Last I heard, you can't step through C lines. That's why gpsim always prints
> "FIXME: HLL files are not supported at the moment" on the terminal.

Yeah that is an important missing feature.

I do not use gpsim that much either. But it is quite unique in the
sense that it can simulate a system. MPSIM does not simulate
a system. KTechlab is another interesting one. It would be nice to see
gpsim and Ktechlab getting more matured.

In the Windows world, it seems quite some people are using
Proteus VSM (they have a MPLAB plugin). I have not used it
but it seems to be quite capable (it is even able to simulate
USB function).

Xiaofan

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...

[Xiaofan Chen]

On Thu, Nov 13, 2008 at 5:18 PM, Vaclav Peroutka <vaclavpe@...> wrote:
>> Piklab is not moving.
> I never used it

It is a nice IDE and supported quite some compilers and programmers. It
has also limited support of ICD 2 debugging.
http://piklab.sourceforge.net/index.php


>> pk2cmd works fine.
> That is true. But I really miss chip autodetection mechanism
> similar to Windows' PicKit2 programmer. And I do not know how
> to show revision of the PIC. Maybe nice-to-have: automatically
> show ID and revision of the target PIC every time pk2cmd is run.

Future pk2 from Jeff Post may have features.
On the other hand, you can put your feature request for pk2cmd
in this thread. The developer is very responsive.
http://forum.microchip.com/tm.aspx?m=260540


>> Still no good debugger support.
> That is definitely true. Does ANY debugger for PIC on Linux
> exist (even some bad debugger) ? Support of PicKit2 debugger will be really cool.
>

That is the most difficult task as Microchip does not want
to open the debugging protocol.

Xiaofan

---------------------------------------------------------------------
To unsubscribe, e-mail: gnupic-unsubscribe@...
For additional commands, e-mail: gnupic-help@...