Serial data methods

[Q Beukes]

Hey,

I'm not sure what the correct term is here, Ausart? UART?

Either way, we receive data from a Digi ConnectME on the serial
interface. The RCIF is flagged, the interrupt service routine invoked
and the byte read with RCREG. What we do then is put it into a
character buffer until a newline is received, at which time we know
the full "request" or "command" was received (we use the newline as a
terminator, as it makes it easy to debug the program with terminal
software like minicom or netcat.

I was just curious for patterns/idioms around data processing,
techniques people use. It's always good to learn of these things,
because you might just do it in the "not the best way for your
requirements".

One problem I noticed with this technique is the speed at which the
ISR executes. I pasted my complete ISR (the SerialRecordChar is merely
a macro that maps to the correct recording method, which is currently
only the one described above. Further, the __serialRecordChar(c)
method is configured to inline. If this wasn't possible I would embed
it, but since it currently is I prefer to have clearly defined "logic
modules". What i mean by this is that the fact that the character is
recorded in a separate function doesn't affect the performance.

Now, the problem I found here is that if I make the routine a bit
slower, I have found I can complete "crash" the serial interface. One
easy way of doing this is to put a few putchar('X') calls into the
different execution paths of the record method. Then send it more 4 or
more bytes at once. After this the interrupt is never called again for
RCIF, though the program's execution continues without a problem. Even
the timer is still executed.

To further describe exactly how this system communicates on the
serial. We wait for commands/requests, they basically consist of one
or more ascii characters (0x20 up to 0x7e), followed by a terminator
(0x0d or 0x0a). If 2 or more terminators are sent (like on windows
\r\n), then the extra ones are simply ignored). We don't do much error
checking in here, as the system communicates with one entity and if an
invalid command would be received then it will simply be ignored later
on. It is quite fool proof currently and can recover from any invalid
command by simply terminating it and sending a new one. Beyond this,
the device will always communicate with our other devices, so we
safely sacrifice this integrity for performance, integrity which isn't
critical anyway.

So, any advice, hints, idioms or patterns would be appreciated.


void interrupt __kmsIntVector( void )

{

    if (TMR0IF)

    {

            ....
    }



#if defined(SERIAL_ENABLED) && SERIAL_ENABLED == 1
    if (RCIF)

    {

        char c = RCREG;

       

        SerialRecordCharacter(c);

       

        RCIF = 0;

    }

#endif
}



void __serialRecordChar(char c)

{

        if (c == CR || c == LF)

        {

                // first character

                if (__serial.bufIndex == 0)

                {

                        return;

                }

               

                __serial.buf[__serial.bufIndex] = 0;

                __serial.dataAvailable = 1;

                __serial.bufIndex = 0;

        }

        else

        {

                __serial.buf[__serial.bufIndex++] = c;

               

                // buffer is full

                if (__serial.bufIndex == SERIAL_BUFFER_SIZE)

                {

                        __serial.bufIndex = 0;

                }

        }

}

Quintin Beukes
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Jo Scherpenisse]

I think this has something to do with the PIC serial IN buffer, a small
FIFO. When data is received before the receive register is free, a bit in
the control-byte is set, that prevents further interrupts from this serial
input.

Jo Scherpenisse

----- Original Message -----
From: "Quintin Beukes" <quintin@...>
To: "Microcontroller discussion list - Public." <piclist@...>
Sent: Friday, October 09, 2009 12:04 PM
Subject: [PIC] Serial data methods

> Hey,
>
> I'm not sure what the correct term is here, Ausart? UART?
>
> Either way, we receive data from a Digi ConnectME on the serial
> interface. The RCIF is flagged, the interrupt service routine invoked
> and the byte read with RCREG. What we do then is put it into a
> character buffer until a newline is received, at which time we know
> the full "request" or "command" was received (we use the newline as a
> terminator, as it makes it easy to debug the program with terminal
> software like minicom or netcat.
>
> I was just curious for patterns/idioms around data processing,
> techniques people use. It's always good to learn of these things,
> because you might just do it in the "not the best way for your
> requirements".
>
> One problem I noticed with this technique is the speed at which the
> ISR executes. I pasted my complete ISR (the SerialRecordChar is merely
> a macro that maps to the correct recording method, which is currently
> only the one described above. Further, the __serialRecordChar(c)
> method is configured to inline. If this wasn't possible I would embed
> it, but since it currently is I prefer to have clearly defined "logic
> modules". What i mean by this is that the fact that the character is
> recorded in a separate function doesn't affect the performance.
>
> Now, the problem I found here is that if I make the routine a bit
> slower, I have found I can complete "crash" the serial interface. One
> easy way of doing this is to put a few putchar('X') calls into the
> different execution paths of the record method. Then send it more 4 or
> more bytes at once. After this the interrupt is never called again for
> RCIF, though the program's execution continues without a problem. Even
> the timer is still executed.
>
> To further describe exactly how this system communicates on the
> serial. We wait for commands/requests, they basically consist of one
> or more ascii characters (0x20 up to 0x7e), followed by a terminator
> (0x0d or 0x0a). If 2 or more terminators are sent (like on windows
> \r\n), then the extra ones are simply ignored). We don't do much error
> checking in here, as the system communicates with one entity and if an
> invalid command would be received then it will simply be ignored later
> on. It is quite fool proof currently and can recover from any invalid
> command by simply terminating it and sending a new one. Beyond this,
> the device will always communicate with our other devices, so we
> safely sacrifice this integrity for performance, integrity which isn't
> critical anyway.
>
> So, any advice, hints, idioms or patterns would be appreciated.
>
>
> void interrupt __kmsIntVector( void )
>
> {
>
>    if (TMR0IF)
>
>    {
>
>            ....
>    }
>
>
>
> #if defined(SERIAL_ENABLED) && SERIAL_ENABLED == 1
>    if (RCIF)
>
>    {
>
>        char c = RCREG;
>
>
>
>        SerialRecordCharacter(c);
>
>
>
>        RCIF = 0;
>
>    }
>
> #endif
> }
>
>
>
> void __serialRecordChar(char c)
>
> {
>
> if (c == CR || c == LF)
>
> {
>
> // first character
>
> if (__serial.bufIndex == 0)
>
> {
>
> return;
>
> }
>
>
>
> __serial.buf[__serial.bufIndex] = 0;
>
> __serial.dataAvailable = 1;
>
> __serial.bufIndex = 0;
>
> }
>
> else
>
> {
>
> __serial.buf[__serial.bufIndex++] = c;
>
>
>
> // buffer is full
>
> if (__serial.bufIndex == SERIAL_BUFFER_SIZE)
>
> {
>
> __serial.bufIndex = 0;
>
> }
>
> }
>
> }
>
> Quintin Beukes
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist

--------------------------------------------------------------------------------



Geen virus gevonden in het binnenkomende-bericht.
Gecontroleerd door AVG - www.avg.com
Versie: 8.5.421 / Virusdatabase: 270.14.8/2423 - datum van uitgifte:
10/08/09 18:33:00

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Harold Hallikainen-2]

You don't specify which PIC you're using, but the UARTs tend to be
similar. From http://ww1.microchip.com/downloads/en/DeviceDoc/41350C.pdf,
we find:

16.1.2.4 Receive Interrupts
The RCIF interrupt flag bit of the PIR1 register is set
whenever the EUSART receiver is enabled and there is
an unread character in the receive FIFO. The RCIF
interrupt flag bit is read-only, it cannot be set or cleared
by software.

So, the RCIF=0 does nothing since the bit is read only. As others have
pointed out, there is a small FIFO in the UART. Because of this, I'd
change the if(RCIF) to while(RCIF). This will loop the routine that pulls
data from the UART until it is empty.

Finally, I like putting stuff from the UART ISR into a FIFO instead of a
linear buffer. In your non-interrupt code, you can pull data from the FIFO
and put it in the linear buffer for command interpretation at your
leisure. Putting data into the command interpreter buffer in the interrupt
requires the mainline code to act quickly on finding the end of command
before new data comes in and overwrites the command. Or, the command
interpreter could be in the ISR, but I like to keep them small and fast.
Of course, if the other end does not send a new command before getting a
response from this end, dropping directly into the linear buffer would
work fine.

In a recent project (movie theater closed captioning receiver), I have
packets of data coming in at 10kbps. It takes a variable amount of time to
deal with the data (most of the time just sticking it in a linear array,
but at the end of a packet doing a CRC check and then acting on it if
good), so the amount of data in the FIFO between the ISR and the mainline
code grows and shrinks as required so nothing gets lost.

Good luck!

Harold

>
>     if (RCIF)
>
>     {
>
>         char c = RCREG;
>
>
>
>         SerialRecordCharacter(c);
>
>
>
>         RCIF = 0;
>
>     }
>

--
FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
opportunities available!
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Hey,

Thanks a lot for your while(RCIF) tip. I'm already busy starting up my
vmware session to make the change and test it. Exactly the type of
things I was hoping for.

Re. your FIFO tip? How do you implement this fifo?

Quintin Beukes



On Fri, Oct 9, 2009 at 5:17 PM, Harold Hallikainen
<harold@...> wrote:

> You don't specify which PIC you're using, but the UARTs tend to be
> similar. From http://ww1.microchip.com/downloads/en/DeviceDoc/41350C.pdf,
> we find:
>
> 16.1.2.4 Receive Interrupts
> The RCIF interrupt flag bit of the PIR1 register is set
> whenever the EUSART receiver is enabled and there is
> an unread character in the receive FIFO. The RCIF
> interrupt flag bit is read-only, it cannot be set or cleared
> by software.
>
> So, the RCIF=0 does nothing since the bit is read only. As others have
> pointed out, there is a small FIFO in the UART. Because of this, I'd
> change the if(RCIF) to while(RCIF). This will loop the routine that pulls
> data from the UART until it is empty.
>
> Finally, I like putting stuff from the UART ISR into a FIFO instead of a
> linear buffer. In your non-interrupt code, you can pull data from the FIFO
> and put it in the linear buffer for command interpretation at your
> leisure. Putting data into the command interpreter buffer in the interrupt
> requires the mainline code to act quickly on finding the end of command
> before new data comes in and overwrites the command. Or, the command
> interpreter could be in the ISR, but I like to keep them small and fast.
> Of course, if the other end does not send a new command before getting a
> response from this end, dropping directly into the linear buffer would
> work fine.
>
> In a recent project (movie theater closed captioning receiver), I have
> packets of data coming in at 10kbps. It takes a variable amount of time to
> deal with the data (most of the time just sticking it in a linear array,
> but at the end of a packet doing a CRC check and then acting on it if
> good), so the amount of data in the FIFO between the ISR and the mainline
> code grows and shrinks as required so nothing gets lost.
>
> Good luck!
>
> Harold
>
>
>
>
>>
>>     if (RCIF)
>>
>>     {
>>
>>         char c = RCREG;
>>
>>
>>
>>         SerialRecordCharacter(c);
>>
>>
>>
>>         RCIF = 0;
>>
>>     }
>>
>
>
> --
> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
> opportunities available!
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Olin Lathrop]

Quintin Beukes wrote:
> Re. your FIFO tip? How do you implement this fifo?

My PIC development environment (http://www.embedinc.com/pic) includes
template modules for interrupt driven UART I/O with FIFOs.  The app calls
UART_PUT and UART_GET to send and receive bytes.  These routines access the
transmit and receive FIFOs, with the interrupt routine doing the work of
accessing the UART hardware.  In a lot of cases you don't really need a
output FIFO, and you can change UART_PUT to spin on TXIF then write to
TXREG.


********************************************************************
Embed Inc, Littleton Massachusetts, http://www.embedinc.com/products
[phone].  Gold level PIC consultants since 2000.
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Harold Hallikainen-2]

> Quintin Beukes wrote:
>> Re. your FIFO tip? How do you implement this fifo?
>
> My PIC development environment (http://www.embedinc.com/pic) includes
> template modules for interrupt driven UART I/O with FIFOs.  The app calls
> UART_PUT and UART_GET to send and receive bytes.  These routines access
> the
> transmit and receive FIFOs, with the interrupt routine doing the work of
> accessing the UART hardware.  In a lot of cases you don't really need a
> output FIFO, and you can change UART_PUT to spin on TXIF then write to
> TXREG.

There was also an extensive discussion of this in July. See
http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292

Harold


--
FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
opportunities available!
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Thanks again.

Funny enough about an hour before I sent this e-mail I was
contemplating alternatives. The head/tail index was actually something
I though up there (though I was thinking pointers it's technically the
same thing).

Quintin Beukes



On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
<harold@...> wrote:

>
>> Quintin Beukes wrote:
>>> Re. your FIFO tip? How do you implement this fifo?
>>
>> My PIC development environment (http://www.embedinc.com/pic) includes
>> template modules for interrupt driven UART I/O with FIFOs.  The app calls
>> UART_PUT and UART_GET to send and receive bytes.  These routines access
>> the
>> transmit and receive FIFOs, with the interrupt routine doing the work of
>> accessing the UART hardware.  In a lot of cases you don't really need a
>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>> TXREG.
>
> There was also an extensive discussion of this in July. See
> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>
> Harold
>
>
> --
> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
> opportunities available!
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Regarding overwriting the buffer with new data when using a static
buffer. What I do currently is something along the lines of this:

void processCommands()
{
  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
  RCIE = 0;
  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
includes a "silent" extra byte to ensure a 0 terminator is always
available
  RCIE = 1;

  .. process tmpBu ..
}

Though the fifo seems a better option since you don't have to disable
the interrupt even for a moment. The only down side is the extra
overhead of comparing the pointers to check for a wrap around, since I
would still need to copy it into another buffer for the string
comparisons when finding a command. Though it's a minor overhead given
the large benefits for an asynchronous protocol.

Quintin Beukes



On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:

> Thanks again.
>
> Funny enough about an hour before I sent this e-mail I was
> contemplating alternatives. The head/tail index was actually something
> I though up there (though I was thinking pointers it's technically the
> same thing).
>
> Quintin Beukes
>
>
>
> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
> <harold@...> wrote:
>>
>>> Quintin Beukes wrote:
>>>> Re. your FIFO tip? How do you implement this fifo?
>>>
>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>> template modules for interrupt driven UART I/O with FIFOs.  The app calls
>>> UART_PUT and UART_GET to send and receive bytes.  These routines access
>>> the
>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>> accessing the UART hardware.  In a lot of cases you don't really need a
>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>> TXREG.
>>
>> There was also an extensive discussion of this in July. See
>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>
>> Harold
>>
>>
>> --
>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>> opportunities available!
>> --
>> http://www.piclist.com PIC/SX FAQ & list archive
>> View/change your membership options at
>> http://mailman.mit.edu/mailman/listinfo/piclist
>>
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Vitaliy-14]

Quintin,

Any chance you could disable "quoted-printable" for your messages? :)

Circular buffer is the way to go. It is definitely cheaper than using
memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
have a function GetChar() which reads from the buffer, and returns one
character.

You can call GetChar() whenever it's convenient, and stick the character
into your commandBuffer[]. In your GetCommand() function, you return the
commandBuffer if GetChar() returned CR or LF.

commandBuffer is then passed to the ProcessCommand() function, which parses
the data and acts on the command.

Vitaliy


----- Original Message -----
From: "Quintin Beukes" <quintin@...>
To: "Microcontroller discussion list - Public." <piclist@...>
Sent: Friday, October 09, 2009 13:30
Subject: Re: [PIC] Serial data methods


Regarding overwriting the buffer with new data when using a static
buffer. What I do currently is something along the lines of this:

void processCommands()
{
  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
  RCIE = 0;
  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
includes a "silent" extra byte to ensure a 0 terminator is always
available
  RCIE = 1;

  .. process tmpBu ..
}

Though the fifo seems a better option since you don't have to disable
the interrupt even for a moment. The only down side is the extra
overhead of comparing the pointers to check for a wrap around, since I
would still need to copy it into another buffer for the string
comparisons when finding a command. Though it's a minor overhead given
the large benefits for an asynchronous protocol.

Quintin Beukes



On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:

> Thanks again.
>
> Funny enough about an hour before I sent this e-mail I was
> contemplating alternatives. The head/tail index was actually something
> I though up there (though I was thinking pointers it's technically the
> same thing).
>
> Quintin Beukes
>
>
>
> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
> <harold@...> wrote:
>>
>>> Quintin Beukes wrote:
>>>> Re. your FIFO tip? How do you implement this fifo?
>>>
>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>> the
>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>> accessing the UART hardware. In a lot of cases you don't really need a
>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>> TXREG.
>>
>> There was also an extensive discussion of this in July. See
>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>
>> Harold
>>
>>
>> --
>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>> opportunities available!
>> --
>> http://www.piclist.com PIC/SX FAQ & list archive
>> View/change your membership options at
>> http://mailman.mit.edu/mailman/listinfo/piclist
>>
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist 

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Hey,

Disable "quoted-printable" ? I didn't even know Gmail does that -
though it seems they do. Sorry, you must have had some weird source
code displayed there ;>

I noticed you use Outlook Express 6. Doesn't it do quoted-printable decoding?

Re. disabling it, I will look into it.

Quintin Beukes



On Sat, Oct 10, 2009 at 11:39 AM, Vitaliy <piclist@...> wrote:

> Quintin,
>
> Any chance you could disable "quoted-printable" for your messages? :)
>
> Circular buffer is the way to go. It is definitely cheaper than using
> memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
> have a function GetChar() which reads from the buffer, and returns one
> character.
>
> You can call GetChar() whenever it's convenient, and stick the character
> into your commandBuffer[]. In your GetCommand() function, you return the
> commandBuffer if GetChar() returned CR or LF.
>
> commandBuffer is then passed to the ProcessCommand() function, which parses
> the data and acts on the command.
>
> Vitaliy
>
>
> ----- Original Message -----
> From: "Quintin Beukes" <quintin@...>
> To: "Microcontroller discussion list - Public." <piclist@...>
> Sent: Friday, October 09, 2009 13:30
> Subject: Re: [PIC] Serial data methods
>
>
> Regarding overwriting the buffer with new data when using a static
> buffer. What I do currently is something along the lines of this:
>
> void processCommands()
> {
>  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
>  RCIE = 0;
>  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
> includes a "silent" extra byte to ensure a 0 terminator is always
> available
>  RCIE = 1;
>
>  .. process tmpBu ..
> }
>
> Though the fifo seems a better option since you don't have to disable
> the interrupt even for a moment. The only down side is the extra
> overhead of comparing the pointers to check for a wrap around, since I
> would still need to copy it into another buffer for the string
> comparisons when finding a command. Though it's a minor overhead given
> the large benefits for an asynchronous protocol.
>
> Quintin Beukes
>
>
>
> On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:
>> Thanks again.
>>
>> Funny enough about an hour before I sent this e-mail I was
>> contemplating alternatives. The head/tail index was actually something
>> I though up there (though I was thinking pointers it's technically the
>> same thing).
>>
>> Quintin Beukes
>>
>>
>>
>> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
>> <harold@...> wrote:
>>>
>>>> Quintin Beukes wrote:
>>>>> Re. your FIFO tip? How do you implement this fifo?
>>>>
>>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>>> the
>>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>>> accessing the UART hardware. In a lot of cases you don't really need a
>>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>>> TXREG.
>>>
>>> There was also an extensive discussion of this in July. See
>>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>>
>>> Harold
>>>
>>>
>>> --
>>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>>> opportunities available!
>>> --
>>> http://www.piclist.com PIC/SX FAQ & list archive
>>> View/change your membership options at
>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>
>>
>
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

You technique sounds good. Flagged for implementation on Monday :>

Quintin Beukes



On Sat, Oct 10, 2009 at 11:39 AM, Vitaliy <piclist@...> wrote:

> Quintin,
>
> Any chance you could disable "quoted-printable" for your messages? :)
>
> Circular buffer is the way to go. It is definitely cheaper than using
> memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
> have a function GetChar() which reads from the buffer, and returns one
> character.
>
> You can call GetChar() whenever it's convenient, and stick the character
> into your commandBuffer[]. In your GetCommand() function, you return the
> commandBuffer if GetChar() returned CR or LF.
>
> commandBuffer is then passed to the ProcessCommand() function, which parses
> the data and acts on the command.
>
> Vitaliy
>
>
> ----- Original Message -----
> From: "Quintin Beukes" <quintin@...>
> To: "Microcontroller discussion list - Public." <piclist@...>
> Sent: Friday, October 09, 2009 13:30
> Subject: Re: [PIC] Serial data methods
>
>
> Regarding overwriting the buffer with new data when using a static
> buffer. What I do currently is something along the lines of this:
>
> void processCommands()
> {
>  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
>  RCIE = 0;
>  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
> includes a "silent" extra byte to ensure a 0 terminator is always
> available
>  RCIE = 1;
>
>  .. process tmpBu ..
> }
>
> Though the fifo seems a better option since you don't have to disable
> the interrupt even for a moment. The only down side is the extra
> overhead of comparing the pointers to check for a wrap around, since I
> would still need to copy it into another buffer for the string
> comparisons when finding a command. Though it's a minor overhead given
> the large benefits for an asynchronous protocol.
>
> Quintin Beukes
>
>
>
> On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:
>> Thanks again.
>>
>> Funny enough about an hour before I sent this e-mail I was
>> contemplating alternatives. The head/tail index was actually something
>> I though up there (though I was thinking pointers it's technically the
>> same thing).
>>
>> Quintin Beukes
>>
>>
>>
>> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
>> <harold@...> wrote:
>>>
>>>> Quintin Beukes wrote:
>>>>> Re. your FIFO tip? How do you implement this fifo?
>>>>
>>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>>> the
>>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>>> accessing the UART hardware. In a lot of cases you don't really need a
>>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>>> TXREG.
>>>
>>> There was also an extensive discussion of this in July. See
>>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>>
>>> Harold
>>>
>>>
>>> --
>>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>>> opportunities available!
>>> --
>>> http://www.piclist.com PIC/SX FAQ & list archive
>>> View/change your membership options at
>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>
>>
>
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Hey Vitaliy

Re. the transfer-encoding, is this better?

-- sample--
=3D=20 <- this should show as 6 characters
 RCIE = 1; <- and this should show an equal sign
-- sample--

Quintin Beukes



On Sat, Oct 10, 2009 at 6:24 PM, Quintin Beukes <quintin@...> wrote:

> You technique sounds good. Flagged for implementation on Monday :>
>
> Quintin Beukes
>
>
>
> On Sat, Oct 10, 2009 at 11:39 AM, Vitaliy <piclist@...> wrote:
>> Quintin,
>>
>> Any chance you could disable "quoted-printable" for your messages? :)
>>
>> Circular buffer is the way to go. It is definitely cheaper than using
>> memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
>> have a function GetChar() which reads from the buffer, and returns one
>> character.
>>
>> You can call GetChar() whenever it's convenient, and stick the character
>> into your commandBuffer[]. In your GetCommand() function, you return the
>> commandBuffer if GetChar() returned CR or LF.
>>
>> commandBuffer is then passed to the ProcessCommand() function, which parses
>> the data and acts on the command.
>>
>> Vitaliy
>>
>>
>> ----- Original Message -----
>> From: "Quintin Beukes" <quintin@...>
>> To: "Microcontroller discussion list - Public." <piclist@...>
>> Sent: Friday, October 09, 2009 13:30
>> Subject: Re: [PIC] Serial data methods
>>
>>
>> Regarding overwriting the buffer with new data when using a static
>> buffer. What I do currently is something along the lines of this:
>>
>> void processCommands()
>> {
>>  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
>>  RCIE = 0;
>>  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
>> includes a "silent" extra byte to ensure a 0 terminator is always
>> available
>>  RCIE = 1;
>>
>>  .. process tmpBu ..
>> }
>>
>> Though the fifo seems a better option since you don't have to disable
>> the interrupt even for a moment. The only down side is the extra
>> overhead of comparing the pointers to check for a wrap around, since I
>> would still need to copy it into another buffer for the string
>> comparisons when finding a command. Though it's a minor overhead given
>> the large benefits for an asynchronous protocol.
>>
>> Quintin Beukes
>>
>>
>>
>> On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:
>>> Thanks again.
>>>
>>> Funny enough about an hour before I sent this e-mail I was
>>> contemplating alternatives. The head/tail index was actually something
>>> I though up there (though I was thinking pointers it's technically the
>>> same thing).
>>>
>>> Quintin Beukes
>>>
>>>
>>>
>>> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
>>> <harold@...> wrote:
>>>>
>>>>> Quintin Beukes wrote:
>>>>>> Re. your FIFO tip? How do you implement this fifo?
>>>>>
>>>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>>>> the
>>>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>>>> accessing the UART hardware. In a lot of cases you don't really need a
>>>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>>>> TXREG.
>>>>
>>>> There was also an extensive discussion of this in July. See
>>>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>>>
>>>> Harold
>>>>
>>>>
>>>> --
>>>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>>>> opportunities available!
>>>> --
>>>> http://www.piclist.com PIC/SX FAQ & list archive
>>>> View/change your membership options at
>>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>>
>>>
>>
>> --
>> http://www.piclist.com PIC/SX FAQ & list archive
>> View/change your membership options at
>> http://mailman.mit.edu/mailman/listinfo/piclist
>>
>> --
>> http://www.piclist.com PIC/SX FAQ & list archive
>> View/change your membership options at
>> http://mailman.mit.edu/mailman/listinfo/piclist
>>
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

It appears not. It seems the quoted printable is "induced" by
something - as new messages I create seems to be fine. Will
investigate further.

Quintin Beukes



On Sat, Oct 10, 2009 at 6:33 PM, Quintin Beukes <quintin@...> wrote:

> Hey Vitaliy
>
> Re. the transfer-encoding, is this better?
>
> -- sample--
> =3D=20 <- this should show as 6 characters
>  RCIE = 1; <- and this should show an equal sign
> -- sample--
>
> Quintin Beukes
>
>
>
> On Sat, Oct 10, 2009 at 6:24 PM, Quintin Beukes <quintin@...> wrote:
>> You technique sounds good. Flagged for implementation on Monday :>
>>
>> Quintin Beukes
>>
>>
>>
>> On Sat, Oct 10, 2009 at 11:39 AM, Vitaliy <piclist@...> wrote:
>>> Quintin,
>>>
>>> Any chance you could disable "quoted-printable" for your messages? :)
>>>
>>> Circular buffer is the way to go. It is definitely cheaper than using
>>> memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
>>> have a function GetChar() which reads from the buffer, and returns one
>>> character.
>>>
>>> You can call GetChar() whenever it's convenient, and stick the character
>>> into your commandBuffer[]. In your GetCommand() function, you return the
>>> commandBuffer if GetChar() returned CR or LF.
>>>
>>> commandBuffer is then passed to the ProcessCommand() function, which parses
>>> the data and acts on the command.
>>>
>>> Vitaliy
>>>
>>>
>>> ----- Original Message -----
>>> From: "Quintin Beukes" <quintin@...>
>>> To: "Microcontroller discussion list - Public." <piclist@...>
>>> Sent: Friday, October 09, 2009 13:30
>>> Subject: Re: [PIC] Serial data methods
>>>
>>>
>>> Regarding overwriting the buffer with new data when using a static
>>> buffer. What I do currently is something along the lines of this:
>>>
>>> void processCommands()
>>> {
>>>  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
>>>  RCIE = 0;
>>>  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
>>> includes a "silent" extra byte to ensure a 0 terminator is always
>>> available
>>>  RCIE = 1;
>>>
>>>  .. process tmpBu ..
>>> }
>>>
>>> Though the fifo seems a better option since you don't have to disable
>>> the interrupt even for a moment. The only down side is the extra
>>> overhead of comparing the pointers to check for a wrap around, since I
>>> would still need to copy it into another buffer for the string
>>> comparisons when finding a command. Though it's a minor overhead given
>>> the large benefits for an asynchronous protocol.
>>>
>>> Quintin Beukes
>>>
>>>
>>>
>>> On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:
>>>> Thanks again.
>>>>
>>>> Funny enough about an hour before I sent this e-mail I was
>>>> contemplating alternatives. The head/tail index was actually something
>>>> I though up there (though I was thinking pointers it's technically the
>>>> same thing).
>>>>
>>>> Quintin Beukes
>>>>
>>>>
>>>>
>>>> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
>>>> <harold@...> wrote:
>>>>>
>>>>>> Quintin Beukes wrote:
>>>>>>> Re. your FIFO tip? How do you implement this fifo?
>>>>>>
>>>>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>>>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>>>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>>>>> the
>>>>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>>>>> accessing the UART hardware. In a lot of cases you don't really need a
>>>>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>>>>> TXREG.
>>>>>
>>>>> There was also an extensive discussion of this in July. See
>>>>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>>>>
>>>>> Harold
>>>>>
>>>>>
>>>>> --
>>>>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>>>>> opportunities available!
>>>>> --
>>>>> http://www.piclist.com PIC/SX FAQ & list archive
>>>>> View/change your membership options at
>>>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>>>
>>>>
>>>
>>> --
>>> http://www.piclist.com PIC/SX FAQ & list archive
>>> View/change your membership options at
>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>
>>> --
>>> http://www.piclist.com PIC/SX FAQ & list archive
>>> View/change your membership options at
>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>
>>
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Hey,

It seems like gmail replies in quoted-printable whenever the message
contained a line that starts with a space. I contacted them regarding
this.

Quintin Beukes



On Sat, Oct 10, 2009 at 6:34 PM, Quintin Beukes <quintin@...> wrote:

> It appears not. It seems the quoted printable is "induced" by
> something - as new messages I create seems to be fine. Will
> investigate further.
>
> Quintin Beukes
>
>
>
> On Sat, Oct 10, 2009 at 6:33 PM, Quintin Beukes <quintin@...> wrote:
>> Hey Vitaliy
>>
>> Re. the transfer-encoding, is this better?
>>
>> -- sample--
>> =3D=20 <- this should show as 6 characters
>>  RCIE = 1; <- and this should show an equal sign
>> -- sample--
>>
>> Quintin Beukes
>>
>>
>>
>> On Sat, Oct 10, 2009 at 6:24 PM, Quintin Beukes <quintin@...> wrote:
>>> You technique sounds good. Flagged for implementation on Monday :>
>>>
>>> Quintin Beukes
>>>
>>>
>>>
>>> On Sat, Oct 10, 2009 at 11:39 AM, Vitaliy <piclist@...> wrote:
>>>> Quintin,
>>>>
>>>> Any chance you could disable "quoted-printable" for your messages? :)
>>>>
>>>> Circular buffer is the way to go. It is definitely cheaper than using
>>>> memcpy(). Have the ISR put the bytes it receives into the uartBuffer. Then
>>>> have a function GetChar() which reads from the buffer, and returns one
>>>> character.
>>>>
>>>> You can call GetChar() whenever it's convenient, and stick the character
>>>> into your commandBuffer[]. In your GetCommand() function, you return the
>>>> commandBuffer if GetChar() returned CR or LF.
>>>>
>>>> commandBuffer is then passed to the ProcessCommand() function, which parses
>>>> the data and acts on the command.
>>>>
>>>> Vitaliy
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Quintin Beukes" <quintin@...>
>>>> To: "Microcontroller discussion list - Public." <piclist@...>
>>>> Sent: Friday, October 09, 2009 13:30
>>>> Subject: Re: [PIC] Serial data methods
>>>>
>>>>
>>>> Regarding overwriting the buffer with new data when using a static
>>>> buffer. What I do currently is something along the lines of this:
>>>>
>>>> void processCommands()
>>>> {
>>>>  char tmpBuf[SERIAL_BUFFER_SIZE + 1];
>>>>  RCIE = 0;
>>>>  memcpy(tmpBuf, __serial.buf, SERIAL_BUFFER_SIZE + 1); // buffer
>>>> includes a "silent" extra byte to ensure a 0 terminator is always
>>>> available
>>>>  RCIE = 1;
>>>>
>>>>  .. process tmpBu ..
>>>> }
>>>>
>>>> Though the fifo seems a better option since you don't have to disable
>>>> the interrupt even for a moment. The only down side is the extra
>>>> overhead of comparing the pointers to check for a wrap around, since I
>>>> would still need to copy it into another buffer for the string
>>>> comparisons when finding a command. Though it's a minor overhead given
>>>> the large benefits for an asynchronous protocol.
>>>>
>>>> Quintin Beukes
>>>>
>>>>
>>>>
>>>> On Fri, Oct 9, 2009 at 10:20 PM, Quintin Beukes <quintin@...> wrote:
>>>>> Thanks again.
>>>>>
>>>>> Funny enough about an hour before I sent this e-mail I was
>>>>> contemplating alternatives. The head/tail index was actually something
>>>>> I though up there (though I was thinking pointers it's technically the
>>>>> same thing).
>>>>>
>>>>> Quintin Beukes
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 9, 2009 at 9:11 PM, Harold Hallikainen
>>>>> <harold@...> wrote:
>>>>>>
>>>>>>> Quintin Beukes wrote:
>>>>>>>> Re. your FIFO tip? How do you implement this fifo?
>>>>>>>
>>>>>>> My PIC development environment (http://www.embedinc.com/pic) includes
>>>>>>> template modules for interrupt driven UART I/O with FIFOs. The app calls
>>>>>>> UART_PUT and UART_GET to send and receive bytes. These routines access
>>>>>>> the
>>>>>>> transmit and receive FIFOs, with the interrupt routine doing the work of
>>>>>>> accessing the UART hardware. In a lot of cases you don't really need a
>>>>>>> output FIFO, and you can change UART_PUT to spin on TXIF then write to
>>>>>>> TXREG.
>>>>>>
>>>>>> There was also an extensive discussion of this in July. See
>>>>>> http://www.nabble.com/Task-scheduling-in-Embedded-Systems-to24735436.html#a24768292
>>>>>>
>>>>>> Harold
>>>>>>
>>>>>>
>>>>>> --
>>>>>> FCC Rules Updated Daily at http://www.hallikainen.com - Advertising
>>>>>> opportunities available!
>>>>>> --
>>>>>> http://www.piclist.com PIC/SX FAQ & list archive
>>>>>> View/change your membership options at
>>>>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>>>>
>>>>>
>>>>
>>>> --
>>>> http://www.piclist.com PIC/SX FAQ & list archive
>>>> View/change your membership options at
>>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>>
>>>> --
>>>> http://www.piclist.com PIC/SX FAQ & list archive
>>>> View/change your membership options at
>>>> http://mailman.mit.edu/mailman/listinfo/piclist
>>>>
>>>
>>
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Vitaliy-14]

Quintin Beukes wrote:

> Hey Vitaliy
>
> Re. the transfer-encoding, is this better?
>
> -- sample--
> =3D=20 <- this should show as 6 characters
> RCIE = 1; <- and this should show an equal sign
> -- sample--
>
> Quintin Beukes

Yes, this is great -- thank you! OE certainly understands quoted-printable,
but it doesn't use the >s in the quoted text. :(

VItaliy

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

> but it doesn't use the >s in the quoted text. :(

What do you mean with this?

Btw, the reply I sent was still quoted-printable encoded. Just to
recap, I won't be able to turn it off until Gmail fixes the bug.

Q
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Vitaliy-14]

Quintin Beukes wrote:
>> but it doesn't use the >s in the quoted text. :(
>
> What do you mean with this?
>
> Btw, the reply I sent was still quoted-printable encoded. Just to
> recap, I won't be able to turn it off until Gmail fixes the bug.

Q, everything is good now. :)

I meant that before you switched quoted-printable off, OE would not insert
the ">" characters in front of every line of quoted text (like the text I'm
quoting above).

Vitaliy

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Q Beukes]

Must have been something else then. Check my mail headers, qp is still
on. Except for the very last message where I removed all the quoted
e-mails.

Quintin Beukes



On Sat, Oct 10, 2009 at 10:26 PM, Vitaliy <piclist@...> wrote:

> Quintin Beukes wrote:
>>> but it doesn't use the >s in the quoted text. :(
>>
>> What do you mean with this?
>>
>> Btw, the reply I sent was still quoted-printable encoded. Just to
>> recap, I won't be able to turn it off until Gmail fixes the bug.
>
> Q, everything is good now. :)
>
> I meant that before you switched quoted-printable off, OE would not insert
> the ">" characters in front of every line of quoted text (like the text I'm
> quoting above).
>
> Vitaliy
>
> --
> http://www.piclist.com PIC/SX FAQ & list archive
> View/change your membership options at
> http://mailman.mit.edu/mailman/listinfo/piclist
>

--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist

[Vitaliy-14]

Quintin Beukes wrote:
> Must have been something else then. Check my mail headers, qp is still
> on. Except for the very last message where I removed all the quoted
> e-mails.

Whatever you did, it works now.

Vitaliy
--
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist