Create binary from GCJ generated assembly

[isuru herath]

Dear All,

I want to add two assembly instructions to my Java program. Earlier I was doing it in C with "asm volatile". To get the same behavior in Java I used JNI. I used GCJ with -fjni flat to crate the binary. It executed and gave the correct output. But when I checked the memory references between two assembly instructions it is very high compared to the C program.  Thereafter I got the assembly version of both C and Java codes with -S option. In C version, assembly instructions are directly inserted in the proper places. But in Java version library calls are placed in the places where assembly should be inserted. I doubt this may be the case for larger memory references. So my question is is there a way to edit the gcj generated assembly and insert assembly instructions and compile it to a binary. I tried two different ways(trial and error). Nothing worked.

[1]
gcj --main=new_shared_counter -o new_shared_counter new_shared_counter.s

[2]
gcc shared_counter.s

Any help on how to compile a gcj generated assembly to binary would be greatly appreciated.

regards,
Isuru      


     

[Andrew Haley]

isuru herath wrote:

 > I want to add two assembly instructions to my Java program. Earlier
 > I was doing it in C with "asm volatile". To get the same behavior in
 > Java I used JNI. I used GCJ with -fjni flat to crate the binary. It
 > executed and gave the correct output. But when I checked the memory
 > references between two assembly instructions it is very high
 > compared to the C program.  Thereafter I got the assembly version of
 > both C and Java codes with -S option. In C version, assembly
 > instructions are directly inserted in the proper places. But in Java
 > version library calls are placed in the places where assembly should
 > be inserted. I doubt this may be the case for larger memory
 > references. So my question is is there a way to edit the gcj
 > generated assembly and insert assembly instructions and compile it
 > to a binary. I tried two different ways(trial and error). Nothing
 > worked.
 >
 > [1]
 > gcj --main=new_shared_counter -o new_shared_counter new_shared_counter.s
 >
 > [2]
 > gcc shared_counter.s
 >

gcj can't handle inline asm.

There are a few ways to do this:

* Write the code with the asm in C++ using CNI.  This is the easiest
   way, and gives you results identical to using asm volatile in C.

* Add a compiler intrinsic.  This requires specialist knowledge.

* Post-process the assembler output from gcj to replace a call with
   some assembly instructions.  Nasty, but it'd work.  Again, this
   requires specialist knowledge.

 > Any help on how to compile a gcj generated assembly to binary would
 > be greatly appreciated.

Use the assembler, "as".

Andrew.

[isuru herath]

Hi Andrew,

Thanks a lot for the quick reply. I did accordingly. But results were not the same. My program is a multi threaded shared counter.
so in my Java code I have

counter++

I need to surround this with two xchg instructions. like

xchg
counter++
xchg

so that I can count number of read and write operations between two xchg instructions.

with gcc I got both as 1 which is correct.
with gcj and JNI I got 47 read operations and 29 write operations
with gcj and CNI I got 6 read operations and 6 write operations

the way I am doing with CNI is follows.
1. a java class with native method.(custom.java)
2. c++ implementation of it to insert assembly.(custom.cc)
3. compile custom.java (gcj -C custom.java)
4. create the header file (gcjh custom -o custom.h)
5. write multi threaded shared counter program. it has calls to native method to insert assembly(shared_counter.java)
6. compile all java classes (gcj -C *.java)
7. compile the custom.cc (g++ -c custom.cc -o custom.o)
8. create the executable.
(gcj do_options.class custom.class shared_counter.class custom.o -lstdc++ --main=shared_counter -o shared_conter)

do_options is an interface with final variables. is there something that is wrong. or should I need do some additional thing. any help/ advice is greatly appreciated.

regards,
Isuru

--- On Tue, 10/27/09, Andrew Haley <aph@...> wrote:

> From: Andrew Haley <aph@...>
> Subject: Re: Create binary from GCJ generated assembly
> To: "isuru herath" <isuru81@...>
> Cc: java@...
> Date: Tuesday, October 27, 2009, 2:42 AM
> isuru herath wrote:
>
> > I want to add two assembly instructions to my Java
> program. Earlier
> > I was doing it in C with "asm volatile". To get the
> same behavior in
> > Java I used JNI. I used GCJ with -fjni flat to crate
> the binary. It
> > executed and gave the correct output. But when I
> checked the memory
> > references between two assembly instructions it is
> very high
> > compared to the C program.  Thereafter I got the
> assembly version of
> > both C and Java codes with -S option. In C version,
> assembly
> > instructions are directly inserted in the proper
> places. But in Java
> > version library calls are placed in the places where
> assembly should
> > be inserted. I doubt this may be the case for larger
> memory
> > references. So my question is is there a way to edit
> the gcj
> > generated assembly and insert assembly instructions
> and compile it
> > to a binary. I tried two different ways(trial and
> error). Nothing
> > worked.
> >
> > [1]
> > gcj --main=new_shared_counter -o new_shared_counter
> new_shared_counter.s
> >
> > [2]
> > gcc shared_counter.s
> >
>
> gcj can't handle inline asm.
>
> There are a few ways to do this:
>
> * Write the code with the asm in C++ using CNI.  This
> is the easiest
>   way, and gives you results identical to using asm
> volatile in C.
>
> * Add a compiler intrinsic.  This requires specialist
> knowledge.
>
> * Post-process the assembler output from gcj to replace a
> call with
>   some assembly instructions.  Nasty, but it'd
> work.  Again, this
>   requires specialist knowledge.
>
> > Any help on how to compile a gcj generated assembly to
> binary would
> > be greatly appreciated.
>
> Use the assembler, "as".
>
> Andrew.
>
>

     

[Bryce McKinlay-3]

On Wed, Oct 28, 2009 at 11:05 AM, isuru herath <isuru81@...> wrote:

> Thanks a lot for the quick reply. I did accordingly. But results were not the same. My program is a multi threaded shared counter.
> so in my Java code I have
>
> counter++
>
> I need to surround this with two xchg instructions. like
>
> xchg
> counter++
> xchg
>
> so that I can count number of read and write operations between two xchg instructions.
>
> with gcc I got both as 1 which is correct.
> with gcj and JNI I got 47 read operations and 29 write operations
> with gcj and CNI I got 6 read operations and 6 write operations

This is because you are making calls into native code from Java. GCJ
cannot inline native code into your Java code. Calls are not free -
you are measuring the overhead of returning back from native code into
Java code and vice-versa.

If you need to add asm instructions to GCJ compiled code, you can
generate assembler using something like:

gcj -S MyClass.java

Then, edit the resulting assembler .s file to add the instructions.
Finally, link it with something like:

 gcj MyClass.s --main=MyClass

Bryce

[isuru herath]

Hi Bryce,

Thanks for the quick reply. I followed your steps. But got the following error.

shared_counter.s: Assembler messages:
shared_counter.s:1023: Error: file number 1 already allocated

steps I followed.
1. shared counter program with no calls to other classes.
2. generate assembly.(gcj -S shared_counter.java)
3. edit .s file and add instructions.
4. create binary. (gcj shared_counter.s --main=shared_counter)

is there something wrong here or do i need to do another setting. I googled for this error and found lot of bug reports which I couldnot understand. any advice/help is greatly appreciated.

regards,
Isuru



--- On Wed, 10/28/09, Bryce McKinlay <bmckinlay@...> wrote:

> From: Bryce McKinlay <bmckinlay@...>
> Subject: Re: Create binary from GCJ generated assembly
> To: "isuru herath" <isuru81@...>
> Cc: "Andrew Haley" <aph@...>, java@...
> Date: Wednesday, October 28, 2009, 8:22 AM
> On Wed, Oct 28, 2009 at 11:05 AM,
> isuru herath <isuru81@...>
> wrote:
>
> > Thanks a lot for the quick reply. I did accordingly.
> But results were not the same. My program is a multi
> threaded shared counter.
> > so in my Java code I have
> >
> > counter++
> >
> > I need to surround this with two xchg instructions.
> like
> >
> > xchg
> > counter++
> > xchg
> >
> > so that I can count number of read and write
> operations between two xchg instructions.
> >
> > with gcc I got both as 1 which is correct.
> > with gcj and JNI I got 47 read operations and 29 write
> operations
> > with gcj and CNI I got 6 read operations and 6 write
> operations
>
> This is because you are making calls into native code from
> Java. GCJ
> cannot inline native code into your Java code. Calls are
> not free -
> you are measuring the overhead of returning back from
> native code into
> Java code and vice-versa.
>
> If you need to add asm instructions to GCJ compiled code,
> you can
> generate assembler using something like:
>
> gcj -S MyClass.java
>
> Then, edit the resulting assembler .s file to add the
> instructions.
> Finally, link it with something like:
>
>  gcj MyClass.s --main=MyClass
>
> Bryce
>

     

[Andrew Haley]

isuru herath wrote:

> Thanks a lot for the quick reply. I did accordingly. But results were not the same. My program is a multi threaded shared counter.
> so in my Java code I have
>
> counter++
>
> I need to surround this with two xchg instructions. like
>
> xchg
> counter++
> xchg
>
> so that I can count number of read and write operations between two xchg instructions.

OK.

> with gcc I got both as 1 which is correct.
> with gcj and JNI I got 47 read operations and 29 write operations
> with gcj and CNI I got 6 read operations and 6 write operations
>
> the way I am doing with CNI is follows.
> 1. a java class with native method.(custom.java)
> 2. c++ implementation of it to insert assembly.(custom.cc)
> 3. compile custom.java (gcj -C custom.java)
> 4. create the header file (gcjh custom -o custom.h)
> 5. write multi threaded shared counter program. it has calls to native method to insert assembly(shared_counter.java)
> 6. compile all java classes (gcj -C *.java)
> 7. compile the custom.cc (g++ -c custom.cc -o custom.o)
> 8. create the executable.
> (gcj do_options.class custom.class shared_counter.class custom.o -lstdc++ --main=shared_counter -o shared_conter)
>
> do_options is an interface with final variables. is there something that is wrong. or should I need do some additional thing. any help/ advice is greatly appreciated.

I'd rather see your code than guess what you did.

But yes, you're right.  You'd be calling code to invoke the inline asm,
rather than executing it directly.  If you can't tolerate that, your
only recourse is to add an intrinsic counter increment.  gcj already has
intrinsics for things like sun.misc.Unsafe.compareAndSwapInt(), so this one
should be easy enough.  Personally speaking, I'd just write the routines
that need to access the counter in C++.

Andrew.

[isuru herath]

Hi Andrew,

Thanks for the reply. My code is attached.

My intention of using this xchg instruction is that the simulator[Simics] I am using recognizes them as an special event and I can start counting the memory reference from that point onwards. I am trying this on Java is because we are going to use benchmark programs written Java. Thats why I am trying to use CNI to insert assembly to the binary.

I dont understand what you mean by "intrinsic counter". I would greatly appreciate if you could explain it bit more if you have time.

Aslo I didn get clearly what you mean by
 
Personally speaking, I'd just write the routines that need to access the counter in C++.

I am trying to find a way to insert these two assembly instructions to the binary created by gcj so that it will give the same number of memory references as C programs does.

thanks a lot for your help. further help is greatly appreciated.

regards,
isuru

--- On Wed, 10/28/09, Andrew Haley <aph@...> wrote:

> From: Andrew Haley <aph@...>
> Subject: Re: Create binary from GCJ generated assembly
> To: "isuru herath" <isuru81@...>
> Cc: java@...
> Date: Wednesday, October 28, 2009, 8:56 AM
> isuru herath wrote:
>
> > Thanks a lot for the quick reply. I did accordingly.
> But results were not the same. My program is a multi
> threaded shared counter.
> > so in my Java code I have
> > counter++
> >
> > I need to surround this with two xchg instructions.
> like
> >
> > xchg
> > counter++
> > xchg
> >
> > so that I can count number of read and write
> operations between two xchg instructions.
>
> OK.
>
> > with gcc I got both as 1 which is correct.
> > with gcj and JNI I got 47 read operations and 29 write
> operations
> > with gcj and CNI I got 6 read operations and 6 write
> operations
> >
> > the way I am doing with CNI is follows.
> > 1. a java class with native method.(custom.java)
> > 2. c++ implementation of it to insert
> assembly.(custom.cc)
> > 3. compile custom.java (gcj -C custom.java)
> > 4. create the header file (gcjh custom -o custom.h)
> > 5. write multi threaded shared counter program. it has
> calls to native method to insert
> assembly(shared_counter.java)
> > 6. compile all java classes (gcj -C *.java)
> > 7. compile the custom.cc (g++ -c custom.cc -o
> custom.o)
> > 8. create the executable. (gcj do_options.class
> custom.class shared_counter.class custom.o -lstdc++
> --main=shared_counter -o shared_conter)
> >
> > do_options is an interface with final variables. is
> there something that is wrong. or should I need do some
> additional thing. any help/ advice is greatly appreciated.
>
> I'd rather see your code than guess what you did.
>
> But yes, you're right.  You'd be calling code to
> invoke the inline asm,
> rather than executing it directly.  If you can't
> tolerate that, your
> only recourse is to add an intrinsic counter
> increment.  gcj already has
> intrinsics for things like
> sun.misc.Unsafe.compareAndSwapInt(), so this one
> should be easy enough.  Personally speaking, I'd just
> write the routines
> that need to access the counter in C++.
>
> Andrew.
>
>

     
[custom.cc]

---

#define _GNU_SOURCE
#include <gcj/cni.h>
#include <pthread.h>
#include <iostream>
#include "custom.h"
#include "java_magic_c.h"


using namespace std;
using namespace java::lang;

void
custom::do_this(jint option)
{
        MAGIC(option);
}

jint
custom::bind_this_thread(jlong thread_id)
{
// Mask for achieve the hard coded processor affinity
        cpu_set_t mask;
// Initializes all the bits in the mask to zero
        CPU_ZERO(&mask);
// Sets only the bits corresponding to the cpu
        CPU_SET(thread_id, &mask);

        if(sched_setaffinity(0, sizeof(mask), &mask) == -1 )
        {
                cout << "Thread"<< (int)thread_id << ": WARNING: Could not set CPU Affinity, continuing...\n";
        }
        return 0;
}

---

[custom.h]

---

// DO NOT EDIT THIS FILE - it is machine generated -*- c++ -*-

#ifndef __custom__
#define __custom__

#pragma interface

#include <java/lang/Object.h>

extern "Java"
{
  class custom;
}

class custom : public ::java::lang::Object
{
public:
  static void do_this (jint);
  static jint bind_this_thread (jlong);
public: // actually package-private
  custom ();
public:

  static ::java::lang::Class class$;
};

#endif /* __custom__ */

---

[custom.java]

---

class custom
{
        public static native void do_this(int option);
        public static native int bind_this_thread(long thread_id);
}

---

[do_options.java]

---

interface do_options
{
        int TX_ABORT = 0xA;
        int TX_BEGIN = 0xB;
        int TX_COMMIT = 0xC;
        int TX_PRINT = 0xD;

        int RW_PRIVATE = 0x1;
        int RO_SHARED = 0x2;
        int R_SHARED_W_PRIVATE = 0x3;
        int RW_SHARED = 0x4;

        int HEAP = 0x5;

        int THREAD = 0x6;

        int START_SIMULATION = 0x8;
        int STOP_SIMULATION = 0x9;
}

---

[shared_counter.java]

---

class shared_counter implements Runnable, do_options
{
        static int counter = 0;
        long thread_id;
       
        shared_counter(long thread_id)
        {
                this.thread_id = thread_id;
        }

        long get_thread_id()
        {
                return thread_id;
        }

        public void run()
        {
                custom.bind_this_thread(get_thread_id());
                custom.do_this(do_options.TX_BEGIN);
                counter++;
                custom.do_this(do_options.TX_COMMIT);
        }

        public static void main(String[]args)
        {
                Thread sc[] = new Thread[Integer.parseInt(args[0])];
                for(int i = 0;i<sc.length;i++)
                {
                        sc[i] = new Thread(new shared_counter(i));
                        sc[i].start();
                }

                for(int i = 0;i<sc.length;i++)
                {
                        try
                        {
                                sc[i].join();
                        }
                        catch(Exception e)
                        {
                                System.out.println("Got the error "+e);
                        }
                       
                }
                System.out.println("shared counter value is "+counter);
        }
       
}

---

[java_magic_c.h]

---

#ifndef JAVA_MAGIC_C_H_
#define JAVA_MAGIC_C_H_
/* use to allow c++ files to import it */
#ifdef __cplusplus
extern "C" {
#endif

#define ABORT_TX 0xA
#define BEGIN_TX 0xB
#define COMMIT_TX 0xC
#define PRINT_TX 0xD


#define MAGIC(n) do {                                \
  asm volatile ("movl %0, %%eax" : : "g" (n) : "eax");  \
  asm volatile ("xchg %bx,%bx");                        \
} while (0)

#define MAGIC_BREAKPOINT MAGIC(0)

#ifdef __cplusplus
}
#endif

#endif /*JAVA_MAGIC_C_*/

---

[Andrew Haley]

isuru herath wrote:

 > Thanks for the reply. My code is attached.

I'll have a look.

 > My intention of using this xchg instruction is that the
 > simulator[Simics] I am using recognizes them as an special event and
 > I can start counting the memory reference from that point onwards. I
 > am trying this on Java is because we are going to use benchmark
 > programs written Java. Thats why I am trying to use CNI to insert
 > assembly to the binary.
 >
 > I dont understand what you mean by "intrinsic counter". I would
 > greatly appreciate if you could explain it bit more if you have
 > time.

gcj has intrinsics for some functions.  If you want to add an
intrinsic to do your xchg instruction, look at how
sun.misc.Unsafe.compareAndSwapInt() works.  It inserts special
instructions whenever it sees sun.misc.Unsafe.compareAndSwapInt().

 > Aslo I didn get clearly what you mean by
 >
 > Personally speaking, I'd just write the routines that need to access
 > the counter in C++.
 >
 > I am trying to find a way to insert these two assembly instructions
 > to the binary created by gcj so that it will give the same number of
 > memory references as C programs does.

OK.

Andrew.


P.S.  Your mailer is a disaster.  Look at what it did to the layout of
my posting:

 >> I'd rather see your code than guess what you did.
 >>
 >> But yes, you're right.  You'd be calling code to
 >> invoke the inline asm,
 >> rather than executing it directly.  If you can't
 >> tolerate that, your
 >> only recourse is to add an intrinsic counter
 >> increment.  gcj already has
 >> intrinsics for things like
 >> sun.misc.Unsafe.compareAndSwapInt(), so this one
 >> should be easy enough.  Personally speaking, I'd just
 >> write the routines
 >> that need to access the counter in C++.
 >>
 >> Andrew.
 >>
 >>