Larger than 64K memory sections

View: New views
15 Messages — Rating Filter:   Alert me  

Larger than 64K memory sections

by Parthasaradhi Nayani :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello all,
I am using a 4MB flash memory with my Mega64. I would like to create a memory section for the entire 4MB. Is it possible? If yes, how can this be done in the makefile? Thank you very much for your time.

Regards,
Nayani



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Larger than 64K memory sections

by David Brown-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Parthasaradhi Nayani wrote:
> Hello all,
> I am using a 4MB flash memory with my Mega64. I would like to create a
> memory section for the entire 4MB. Is it possible? If yes, how can this
> be done in the makefile? Thank you very much for your time.
>
> Regards,
> Nayani
>

You can't directly access such large memory areas from an AVR, therefore
it makes no sense to talk about a memory section for it.  You have to
manually implement some sort of access to the memory (for example, you
might have the device connected to the standard external memory bus,
which provides the low address bits, then use paging for the higher bits).




_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Jan Waclawek :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

>> I am using a 4MB flash memory with my Mega64. I would like to create a
>> memory section for the entire 4MB. Is it possible? If yes, how can this
>> be done in the makefile? Thank you very much for your time.
>>
>
>You can't directly access such large memory areas from an AVR, therefore
>it makes no sense to talk about a memory section for it.

IMHO your implication is incorrect. Even if there's no builtin support for non-RAM variables in avr-gcc (yet), we already have sections for EEPROM and FLASH - and the latter can be >64kB large.

JW



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Larger than 64K memory sections

by David Brown-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jan Waclawek wrote:

>>> I am using a 4MB flash memory with my Mega64. I would like to
>>> create a memory section for the entire 4MB. Is it possible? If
>>> yes, how can this be done in the makefile? Thank you very much
>>> for your time.
>>>
>> You can't directly access such large memory areas from an AVR,
>> therefore it makes no sense to talk about a memory section for it.
>
> IMHO your implication is incorrect. Even if there's no builtin
> support for non-RAM variables in avr-gcc (yet), we already have
> sections for EEPROM and FLASH - and the latter can be >64kB large.
>

Yes, that's true.  I hadn't thought about the EEPROM sections, mainly
because I think they are often a bad idea in practice and thus don't use
them (it's far too easy for the final addresses to change between
builds, making a mess of stored data after changing the software.  I
much prefer to use a struct at a fixed manually-specified address to
order my eeprom data).

Still, without direct access the OP will need manual paging at least
(and possibly more manual work - for all we know, this could be a serial
flash, or 16-bit flash connected via GPIOs, or whatever).

Memory sections and pointer arithmetic get messy or impossible after 32K
or 64K limits.  A 4MB section may also crash with other sections in the
linker file unless you are very careful.

All in all, I can't see any advantage to the OP in using memory sections
here, and I can see a great many complications if it can be made to work
at all.



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by David Kelly :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, Aug 26, 2009 at 03:10:05PM +0200, David Brown wrote:
>
> All in all, I can't see any advantage to the OP in using memory
> sections here, and I can see a great many complications if it can be
> made to work at all.

I agree that with the avr-gcc toolset that a great many complications
will occur if one tries to map 4MB into the existing 4MB memory space
used by the linker. For lack of multiple address spaces the AVR tools
magically assume an address space way out in never-never land is EEPROM.
Perhaps the linker can be made 64-bit with minimal effort?

I disagree with the claim that mapping external regions like this in to
linker address space is a bad idea. We know nothing of the user's
application or design for utilizing this space. Is very useful to create
named globals and let the linker assign the address whether or not this
global is in FLASH, RAM, EEPROM, or an external device. Otherwise one
must manually manage allocation.

--
David Kelly N4HHE, dkelly@...
========================================================================
Whom computers would destroy, they must first drive mad.


_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Parthasaradhi Nayani :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


--- On Wed, 8/26/09, David Brown <david@...> wrote:

From: David Brown <david@...>
Subject: [avr-gcc-list] Re: Larger than 64K memory sections
To: avr-gcc-list@...
Date: Wednesday, August 26, 2009, 6:40 PM

> sections for EEPROM and FLASH - and the latter can be >64kB large.

Yes, that's true.  I hadn't thought about the EEPROM sections,

Hello,
Perhaps as David Brown mentioned it may be better to manually handle larger memories, but I would like to know how larger sections are defined in the makefile? Any pointers will be appreciated. Thank you.

Nayani



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Larger than 64K memory sections

by David Brown-40 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

David Kelly wrote:

> On Wed, Aug 26, 2009 at 03:10:05PM +0200, David Brown wrote:
>> All in all, I can't see any advantage to the OP in using memory
>> sections here, and I can see a great many complications if it can be
>> made to work at all.
>
> I agree that with the avr-gcc toolset that a great many complications
> will occur if one tries to map 4MB into the existing 4MB memory space
> used by the linker. For lack of multiple address spaces the AVR tools
> magically assume an address space way out in never-never land is EEPROM.
> Perhaps the linker can be made 64-bit with minimal effort?
>

A 32-bit address space has 4 GB, so there is no need to go to 64-bit.
But the avr-gcc tools already have some "magic" to fit a 16-bit data
space, 16-bit eeprom space, and >16-bit flash space at different
addresses within a virtual 32-bit address space that does not really
exist on the avr.  Adding new large address spaces may be easy, or it
may be complicated - I have not tried this myself, but I it would at
least require some effort to understand the existing system and see what
is involved.

> I disagree with the claim that mapping external regions like this in to
> linker address space is a bad idea. We know nothing of the user's
> application or design for utilizing this space. Is very useful to create
> named globals and let the linker assign the address whether or not this
> global is in FLASH, RAM, EEPROM, or an external device. Otherwise one
> must manually manage allocation.
>

You are right that we know nothing about the application in question -
details here would make it much easier to give recommendations.  But for
most purposes, standard allocation of global data in memory spaces is
only suitable for code flash and ram data - not eeprom, external flash,
or external devices.  The problem is that the compiler and/or linker are
free to re-order and re-arrange any objects they allocate.

Suppose you've made a payment card application.  In one place, you've
got an eeprom-allocated variable "moneyLeft", and in another place
you've got "runTimeSeconds".  When re-compiling after a change, the
compiler/linker could easily swap these two - updating software from
version 1.00.01 to 1.00.02 would then trash your eeprom data.

It does not matter what addresses are used for ram variables, nor for
the flash code that is updated.  But very often it /does/ matter for
persistent data, and it is certainly important for pre-defined memory
maps such as for external peripherals (there is a good reason why
internal peripherals are not defined in a file with lines like "volatile
uint8_t PORTD" !).  The only way to be entirely sure that an object is
allocated to the same address each time is for it to be the only thing
in a section, and the section is given an explicit address at link time.
  When you are talking about data that can't be addressed directly in C
on the AVR anyway, why bother with that?

Note that you /can/ (and should) get the compiler to do most of the
effort in the allocation.  The easiest way is to make a struct type for
all your eeprom data:

        typedef struct { uint16_t moneyLeft; uint16_t runTimeSeconds; }
        eepromData;

Then you can access this data with something like:

        eeprom_read_word((uint16_t*) (baseAddress +
                        offsetof(eepromData, runTimeSeconds)));

You manually specify a base address, and let the compiler handle the rest.

The same sort of system will work for external data, although I would
guess that the 4 MB flash will be filled with an array or other such
structure - it would be easier to fit the details within getter and
setter access functions.  Typedef'ed structs, offsetof and sizeof will
probably still feature in the code - precisely to avoid manual allocation.



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Parthasaradhi Nayani :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

David Kelly wrote:
You are right that we know nothing about the application in question - details here would make it much easier to give recommendations.

Hello all,

I mentioned 4MB flash as it is the HW spec. This 4MB gets filled over a period of time.

Details of application - I have data packets each 32 bytes in size and we have accommodate 2000 of these (predefined). This I plan to put in a part of the 4MB Flash. Rest of the memory will be used for storing records comprising of these data packets with data and time etc. The records will be deleted from time to time but the data packets will remain. It is possible to use a long int as an address pointer from the start of the available memory (after the data packets) and store records. I still curious to know how one can create a section larger than 64K.

Nayani

start: 0000-00-00 end: 0000-00-00
_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Tero Sinervo :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Parthasaradhi Nayani wrote:
> Details of application - I have data packets each 32 bytes in size and we have accommodate 2000 of these (predefined). This I plan to put in a part of the 4MB Flash. Rest of the memory will be used for storing records comprising of these data packets with data and time etc. The records will be deleted from time to time but the data packets will remain. It is possible to use a long int as an address pointer from the start of the available memory (after the data packets) and store records. I still curious to know how one can create a section larger than 64K.

You'll have to customise your linker script and add your section there.
I fiddled with this a while ago but sadly have lost the results both in
computer and brain memory. After the linker knows what you want to do
you'll be a bit farther down the road.

Then you'll have to deal with pointers being reduced to 16 bits. You
should see Carlos Lamas's morepgmspace.h. It might give you some help
with that.


--
Tero Sinervo



_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Tero Sinervo :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

David Brown wrote:

> It does not matter what addresses are used for ram variables, nor for
> the flash code that is updated.  But very often it /does/ matter for
> persistent data, and it is certainly important for pre-defined memory
> maps such as for external peripherals (there is a good reason why
> internal peripherals are not defined in a file with lines like "volatile
> uint8_t PORTD" !).  The only way to be entirely sure that an object is
> allocated to the same address each time is for it to be the only thing
> in a section, and the section is given an explicit address at link time.
>  When you are talking about data that can't be addressed directly in C
> on the AVR anyway, why bother with that?

Yes, very often but not always. Consider having text strings in pgmspace
but running out of space. Good thing you have a mass storage and file
system - you can move the text strings to an external section and load
the section to a file. Only thing you need to do is write a driver that
replaces pgmspace string stuff with file access. It doesn't matter in
which order the strings are in the file.


> Note that you /can/ (and should) get the compiler to do most of the
> effort in the allocation.  The easiest way is to make a struct type for
> all your eeprom data:
>
>     typedef struct { uint16_t moneyLeft; uint16_t runTimeSeconds; }
>     eepromData;
>
> Then you can access this data with something like:
>
>     eeprom_read_word((uint16_t*) (baseAddress +
>             offsetof(eepromData, runTimeSeconds)));
>
> You manually specify a base address, and let the compiler handle the rest.

This is all all right although perhaps offsetof is redundant:

eepromData* eeprom = BASEADDRESS;
eeprom_read_word(&eeprom->runTimeSeconds);


--
Tero Sinervo


_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Larger than 64K memory sections

by David Brown-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Tero Sinervo wrote:

> David Brown wrote:
>> It does not matter what addresses are used for ram variables, nor for
>> the flash code that is updated.  But very often it /does/ matter for
>> persistent data, and it is certainly important for pre-defined memory
>> maps such as for external peripherals (there is a good reason why
>> internal peripherals are not defined in a file with lines like
>> "volatile uint8_t PORTD" !).  The only way to be entirely sure that an
>> object is allocated to the same address each time is for it to be the
>> only thing in a section, and the section is given an explicit address
>> at link time.  When you are talking about data that can't be addressed
>> directly in C on the AVR anyway, why bother with that?
>
> Yes, very often but not always. Consider having text strings in pgmspace
> but running out of space. Good thing you have a mass storage and file
> system - you can move the text strings to an external section and load
> the section to a file. Only thing you need to do is write a driver that
> replaces pgmspace string stuff with file access. It doesn't matter in
> which order the strings are in the file.
>

That's a good example, and in general if the external data is updated
with the software, then there is no need to be consistent about the
addresses.  In that case an external section can do the job.

But you want to be very sure that the external data /is/ consistent -
I've worked with systems where the text strings were in an external
eeprom, which should have been updated with the software.  Since (by
mistake) they were not updated, test systems started showing messages in
the wrong language...

I often find that when you want to generate data for external memory
like this, a C compiler is not the best tool for the job in the first place.

>
>> Note that you /can/ (and should) get the compiler to do most of the
>> effort in the allocation.  The easiest way is to make a struct type
>> for all your eeprom data:
>>
>>     typedef struct { uint16_t moneyLeft; uint16_t runTimeSeconds; }
>>     eepromData;
>>
>> Then you can access this data with something like:
>>
>>     eeprom_read_word((uint16_t*) (baseAddress +
>>             offsetof(eepromData, runTimeSeconds)));
>>
>> You manually specify a base address, and let the compiler handle the
>> rest.
>
> This is all all right although perhaps offsetof is redundant:
>
> eepromData* eeprom = BASEADDRESS;
> eeprom_read_word(&eeprom->runTimeSeconds);
>

Yes, that's a neater way to write it in this case.  Writing the sums
manually (with offsetof) is perhaps easier to adapt to 32-bit addresses
for an external 4 MB flash, and also possibly easier to relocate to
different base addresses.  It all gets buried away in accessor functions
or macros, so you should write whatever way you think is clearest.




_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by David Kelly :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wed, Aug 26, 2009 at 08:21:00PM -0700, Parthasaradhi Nayani wrote:
> David Kelly wrote:
> > You are right that we know nothing about the application in question
> > - details here would make it much easier to give recommendations.
>
> Hello all,
>
> I mentioned 4MB flash as it is the HW spec. This 4MB gets filled over
> a period of time.

4 million bytes or bits? Most FLASH devices are sized by the bit.

> Details of application - I have data packets each 32 bytes in size and
> we have accommodate 2000 of these (predefined).

32 * 2000 = 6400 bytes.

> This I plan to put in a part of the 4MB Flash. Rest of the memory will
> be used for storing records comprising of these data packets with data
> and time etc. The records will be deleted from time to time but the
> data packets will remain. It is possible to use a long int as an
> address pointer from the start of the available memory (after the data
> packets) and store records. I still curious to know how one can create
> a section larger than 64K.

What you describe is probably best *not* mapped into avr-gcc name space or
AVR address space. Is probably best to #define the starting offsets of
each of your blocks of data.

No matter what you do you won't be able to say *MyUInt16Ptr = 0x1234;
you will have to write a routine and use it something like this:

u16_value = 0x1234;
result = Write4MBFlash( (uint32_t)MyUint16Ptr, sizeof(uint16_t), &u16_value );

All the external SPI FLASH devices I have used are paged. One
read/writes to a RAM buffer in the device then flush that buffer to a
page in FLASH. IIRC the Atmel device I last used had 264 byte pages
which served to complicate addressing if one insisted on using all 264
bytes. Believe the designer's intent was that one use 256 bytes as a
"sector" and the other 8 bytes for tracking usage, even linking to
previous sector and next sector for a form of filesystem.

Atmel had a very useful appnote containing C routines for access to
their DataFlash parts which helped a lot. I felt the need to heavily
edit it for style and naming convention. Rearranged a bit, and deleted a
lot that I didn't use. But it was a very good start.

--
David Kelly N4HHE, dkelly@...
========================================================================
Whom computers would destroy, they must first drive mad.


_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by David Kelly :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Thu, Aug 27, 2009 at 09:55:05AM -0500, David Kelly wrote:
> On Wed, Aug 26, 2009 at 08:21:00PM -0700, Parthasaradhi Nayani wrote:
>
> > Details of application - I have data packets each 32 bytes in size and
> > we have accommodate 2000 of these (predefined).
>
> 32 * 2000 = 6400 bytes.

Arrgh! Decaf!

64,000 bytes. Half of a megabit.

--
David Kelly N4HHE, dkelly@...
========================================================================
Whom computers would destroy, they must first drive mad.


_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

RE: Re: Larger than 64K memory sections

by Stu Bell :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> On Thu, Aug 27, 2009 at 09:55:05AM -0500, David Kelly wrote:
> > On Wed, Aug 26, 2009 at 08:21:00PM -0700, Parthasaradhi
> Nayani wrote:
> >
> > > Details of application - I have data packets each 32 bytes in size

> > > and we have accommodate 2000 of these (predefined).
> >
> > 32 * 2000 = 6400 bytes.
>
> Arrgh! Decaf!
>
> 64,000 bytes. Half of a megabit.

:D  must...   find....   antidote....

Stu


_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list

Re: Re: Larger than 64K memory sections

by Parthasaradhi Nayani :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

From: David Kelly <dkelly@...>

>4 million bytes or bits? Most FLASH devices are sized by the bit.

It is 4M Bytes.


>No matter what you do you won't be able to say *MyUInt16Ptr = 0x1234;
>you will have to write a routine and use it something like this:

>u16_value = 0x1234;
>result = Write4MBFlash( (uint32_t)MyUint16Ptr, sizeof(uint16_t), &u16_value );

One advantage I found, using name spaces (sections) is, generating a hex file for initialised data, which can be downloaded through serial port.

>All the external SPI FLASH devices I have used are paged. One
>read/writes to a RAM buffer in the device then flush that buffer to a
>page in FLASH. IIRC the Atmel device I last used had 264 byte pages
>which served to complicate addressing if one insisted on using all 264
>bytes. Believe the designer's intent was that one use 256 bytes as a
>"sector" and the other 8 bytes for tracking usage, even linking to
>previous sector and next sector for a form of filesystem.

Never thought of this (using the extra 8 bytes for tracking/linking to other sectors)

>Atmel had a very useful appnote containing C routines for access to
>their DataFlash parts which helped a lot. I felt the need to heavily
>edit it for style and naming convention. Rearranged a bit, and deleted a
>lot that I didn't use. But it was a very good start.

Will check Atmel's site . Thank you.

Nayani


start: 0000-00-00 end: 0000-00-00
_______________________________________________
AVR-GCC-list mailing list
AVR-GCC-list@...
http://lists.nongnu.org/mailman/listinfo/avr-gcc-list