AT91SAM7X256 TC1 Timer being disabled

View: New views
9 Messages — Rating Filter:   Alert me  

AT91SAM7X256 TC1 Timer being disabled

by Coleman Brumley-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello everyone,

 

In my application, I've enabled the TC1 timer using the following code:

 

void RegisterTC1Timer(void (*handler) (void *))

{

    int dummy;

 

#if defined(MCU_AT91SAM7X256) || defined(MCU_AT91SAM9260)

    /* Enable TC1 clock. */

    outr(PMC_PCER, _BV(TC1_ID));

#endif

 

    /* Disable the Clock Counter */

    outr(TC1_CCR, TC_CLKDIS);

    /* Disable all interrupts */

    outr(TC1_IDR, 0xFFFFFFFF);

    /* Clear the status register. */

    dummy = inr(TC1_SR);

    /* Select divider and compare trigger */

    outr(TC1_CMR, TC_CLKS_MCK32 | TC_CPCTRG);

    /* Enable the Clock counter */

    outr(TC1_CCR, TC_CLKEN);

    /* Validate the RC compare interrupt */

    outr(TC1_IER, TC_CPCS);

 

    /* Register timer interrupt handler. */

    NutRegisterIrqHandler(&sig_TC1, handler, 0);

    /* Set to lowest priority. */

    NutIrqSetPriority(&sig_TC1, 0);

 

    /* Enable timer 0 interrupts */

    NutIrqEnable(&sig_TC1);

    //outr(AIC_IECR, _BV(TC1_ID));

 

#ifndef NUT_TICK_FREQ

#define NUT_TICK_FREQ   1000UL

#endif

    /* Set compare value for 1 ms. */

#if defined(AT91_PLL_MAINCK)      

    outr(TC1_RC, At91GetMasterClock() / (32 * NUT_TICK_FREQ));

#else

    outr(TC1_RC, NutGetCpuClock() / (32 * NUT_TICK_FREQ));

#endif

 

    /* Software trigger starts the clock. */

    outr(TC1_CCR, TC_SWTRG);

}

 

I then call RegisterTC1Timer(TC1TimerIntr); in some initialization code at
startup.  This function is pretty simple:

 

static void TC1TimerIntr(void *arg)

{      

       tc1_ticks++;  

       if(tc1ticks>=5)      //do something every 5ms

       {

              //do sometthing, set a flag, toggle a PIO pin, etc.

              tclticks=0;

       }      

}

 

What I'm finding, though, is that after some amount of time (several hours,
for instance) the TC1 interrupt is being disabled.  If I toggle a PIO pin in
TC1TimerIntr and the app gets in this state, then the PIO pin is not longer
being toggled.   I can find no reference to NutIrqDisable for sig_TC1 in the
code though.  

 

Has anyone else come across this type of scenario?  I'm using Nut/OS v4.6.4.


 

Best Regards,

Coleman

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Parent Message unknown Re: AT91SAM7X256 TC1 Timer being disabled

by Coleman Brumley-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



> -----Original Message-----
> From: Bernd Walter [mailto:nut@...]
> Sent: Thursday, July 09, 2009 10:05 AM
> To: cbrumley@...; Ethernut User Chat (English)
> Subject: Re: [En-Nut-Discussion] AT91SAM7X256 TC1 Timer being disabled
>
> On Wed, Jul 08, 2009 at 04:34:39PM -0400, Coleman Brumley wrote:
> > I then call RegisterTC1Timer(TC1TimerIntr); in some initialization
> code at
> > startup.  This function is pretty simple:
> >
> >
> >
> > static void TC1TimerIntr(void *arg)
> >
> > {
> >
> >        tc1_ticks++;
> >
> >        if(tc1ticks>=5)      //do something every 5ms
> >
> >        {
> >
> >               //do sometthing, set a flag, toggle a PIO pin, etc.
> >
> >               tclticks=0;
> >
> >        }
> >
> > }
> >
> >
> >
> > What I'm finding, though, is that after some amount of time (several
> hours,
> > for instance) the TC1 interrupt is being disabled.  If I toggle a PIO
> pin in
> > TC1TimerIntr and the app gets in this state, then the PIO pin is not
> longer
> > being toggled.   I can find no reference to NutIrqDisable for sig_TC1
> in the
> > code though.
>
> I don't know how ethernuts registration works.
> Normally an ISR must clear the interrupt in the AIC at the end.
> I always handle it myself.
>
> AT91F_AIC_ConfigureIt(AT91C_ID_UDP, 5,
> AT91C_AIC_SRCTYPE_INT_HIGH_LEVEL, (void(*)(void)) ISR_UDP);
>
> void ISR_UDP( void ) __attribute__ ((naked));
>
> void
> ISR_UDP( void )
> {
>         IRQ_ENTRY();
>         // TODO
>         IRQ_EXIT();
> }
>
>
>
> AT91F_AIC_ConfigureIt is a macro originating from Atmel:
>
> #define AT91F_AIC_ConfigureIt( irq_id, priority, src_type, newHandler )
> \
> {
> \
>     unsigned int mask ;
> \
>
> \
>     mask = 0x1 << irq_id;
> \
>     /* Disable the interrupt on the interrupt controller */
> \
>     AT91C_BASE_AIC->AIC_IDCR = mask ;
> \
>     /* Save the interrupt handler routine pointer and the interrupt
> priority */ \
>     AT91C_BASE_AIC->AIC_SVR[irq_id] = (unsigned int) newHandler ;
> \
>     /* Store the Source Mode Register */
> \
>     AT91C_BASE_AIC->AIC_SMR[irq_id] = src_type | priority  ;
> \
>     /* Clear the interrupt on the interrupt controller */
> \
>     AT91C_BASE_AIC->AIC_ICCR = mask ;
> \
> }

My understanding of the Nut/OS timer implementation is that it does this
already.  Am I missing something?

My ISR shouldn't use NutIrqDisable(&sig_TC1) and then call
NutIrqEnable(&sig_TC1) upon exit should it?  If that were the case, then the
timer interrupt shouldn't work at all...however, in my case it works for
several hours (but never the same amount of several hours) before becoming
disabled.  

Would this have something to do with the use of
NutEnterCritcal/NutExitCritical which is now frowned upon?

- Coleman

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Tim DeBaillie :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Coleman Brumley wrote:

> What I'm finding, though, is that after some amount of time (several hours,
> for instance) the TC1 interrupt is being disabled.  If I toggle a PIO pin in
> TC1TimerIntr and the app gets in this state, then the PIO pin is not longer
> being toggled.   I can find no reference to NutIrqDisable for sig_TC1 in the
> code though.  
>
>  
>
> Has anyone else come across this type of scenario?  I'm using Nut/OS v4.6.4.
>  
This is a problem we have seen on many occasions.  The problem lies in
that Ethernut does not allow re-entrance of IRQs.  If you are to get two
interrupts on your timer before you are able to handle them, then it
causes the timer to just stop.

The REAL solution is to rewrite the IRQ_ENTRY and IRQ_EXIT to allow for
re-entrance of IRQs (and not disable IRQs).

Our temporary solution involves using a wrapping timer instead of a
resetting timer.  Additionally, if the timer's accuracy is highly
important, you should check for overflow and handle it (ie: two
interrupts before handler, should be detectable by amount in TC1_CV)

// CODE

#define COMPARE_ADDER 0xdc8
signed short compare_value = COMPARE_ADDER;

static void RealTimeClock(void *arg){

   local_clock++;

   //you could check for overflow here

   //increase compare value to next interrupt value
   //(signed short allows for overflow wrapping)
   compare_value += COMPARE_ADDER;
   outr(TC1_RC, compare_value);

}

void InitTimer1(void){

   //change the AIC to allow handle of TC1 interrupt
   outr(AIC_IECR, _BV(TC1_ID));

   //enable timer1 clock
   outr(PMC_PCER, _BV(TC1_ID));

   //disable the timer
   outr(TC1_CCR, TC_CLKDIS);

   //disable interrupts on the timer
   outr(TC1_IDR, 0xFFFFFFFF);

   //reading the status register will clear any pending interrupt
   inr(TC1_SR);

   //Select MCK/32, inc on + edge (45.1584MHz/32 = 1.4112MHz)
   outr(TC1_CMR, TC_CLKS_MCK32);

   //set compare value to 2.50ms (.0025 * 1411200) = 3528 (0xdc8)
   outr(TC1_RC, COMPARE_ADDER);

   //enable the timer
   outr(TC1_CCR, TC_CLKEN);

   //enable the rc compare interrupts
   outr(TC1_IER, TC_CPCS);

   //register the interrupt
   NutRegisterIrqHandler(&sig_TC1, RealTimeClock, 0);

   //set to highest priority (7)
   NutIrqSetPriority(&sig_TC1, 7);

   //enable the interrupt
   NutIrqEnable(&sig_TC1);

   //reset the counter and start the clock
   outr(TC1_CCR, TC_SWTRG);

}

//END CODE

Hope this helps,

Tim DeBaillie
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Coleman Brumley-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Tim,

Thank you.  I'm testing this code now in my application...so far so good.  

The last test took almost 12 hours to fail though, so I won't know for sure
until tomorrow.

- Coleman


> -----Original Message-----
> From: en-nut-discussion-bounces@... [mailto:en-nut-discussion-
> bounces@...] On Behalf Of Timothy M. De Baillie
> Sent: Friday, July 10, 2009 12:13 PM
> To: Ethernut User Chat (English)
> Subject: Re: [En-Nut-Discussion] AT91SAM7X256 TC1 Timer being disabled
>
> Coleman Brumley wrote:
> > What I'm finding, though, is that after some amount of time (several
> hours,
> > for instance) the TC1 interrupt is being disabled.  If I toggle a PIO
> pin in
> > TC1TimerIntr and the app gets in this state, then the PIO pin is not
> longer
> > being toggled.   I can find no reference to NutIrqDisable for sig_TC1
> in the
> > code though.
> >
> >
> >
> > Has anyone else come across this type of scenario?  I'm using Nut/OS
> v4.6.4.
> >
> This is a problem we have seen on many occasions.  The problem lies in
> that Ethernut does not allow re-entrance of IRQs.  If you are to get
> two
> interrupts on your timer before you are able to handle them, then it
> causes the timer to just stop.
>
> The REAL solution is to rewrite the IRQ_ENTRY and IRQ_EXIT to allow for
> re-entrance of IRQs (and not disable IRQs).
>
> Our temporary solution involves using a wrapping timer instead of a
> resetting timer.  Additionally, if the timer's accuracy is highly
> important, you should check for overflow and handle it (ie: two
> interrupts before handler, should be detectable by amount in TC1_CV)
>
> // CODE
>
> #define COMPARE_ADDER 0xdc8
> signed short compare_value = COMPARE_ADDER;
>
> static void RealTimeClock(void *arg){
>
>    local_clock++;
>
>    //you could check for overflow here
>
>    //increase compare value to next interrupt value
>    //(signed short allows for overflow wrapping)
>    compare_value += COMPARE_ADDER;
>    outr(TC1_RC, compare_value);
>
> }
>
> void InitTimer1(void){
>
>    //change the AIC to allow handle of TC1 interrupt
>    outr(AIC_IECR, _BV(TC1_ID));
>
>    //enable timer1 clock
>    outr(PMC_PCER, _BV(TC1_ID));
>
>    //disable the timer
>    outr(TC1_CCR, TC_CLKDIS);
>
>    //disable interrupts on the timer
>    outr(TC1_IDR, 0xFFFFFFFF);
>
>    //reading the status register will clear any pending interrupt
>    inr(TC1_SR);
>
>    //Select MCK/32, inc on + edge (45.1584MHz/32 = 1.4112MHz)
>    outr(TC1_CMR, TC_CLKS_MCK32);
>
>    //set compare value to 2.50ms (.0025 * 1411200) = 3528 (0xdc8)
>    outr(TC1_RC, COMPARE_ADDER);
>
>    //enable the timer
>    outr(TC1_CCR, TC_CLKEN);
>
>    //enable the rc compare interrupts
>    outr(TC1_IER, TC_CPCS);
>
>    //register the interrupt
>    NutRegisterIrqHandler(&sig_TC1, RealTimeClock, 0);
>
>    //set to highest priority (7)
>    NutIrqSetPriority(&sig_TC1, 7);
>
>    //enable the interrupt
>    NutIrqEnable(&sig_TC1);
>
>    //reset the counter and start the clock
>    outr(TC1_CCR, TC_SWTRG);
>
> }
>
> //END CODE
>
> Hope this helps,
>
> Tim DeBaillie
> _______________________________________________
> http://lists.egnite.de/mailman/listinfo/en-nut-discussion

_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Bernd Walter-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Fri, Jul 10, 2009 at 11:12:41AM -0500, Timothy M. De Baillie wrote:

> Coleman Brumley wrote:
> > What I'm finding, though, is that after some amount of time (several hours,
> > for instance) the TC1 interrupt is being disabled.  If I toggle a PIO pin in
> > TC1TimerIntr and the app gets in this state, then the PIO pin is not longer
> > being toggled.   I can find no reference to NutIrqDisable for sig_TC1 in the
> > code though.  
> >
> >  
> >
> > Has anyone else come across this type of scenario?  I'm using Nut/OS v4.6.4.
> >  
> This is a problem we have seen on many occasions.  The problem lies in
> that Ethernut does not allow re-entrance of IRQs.  If you are to get two
> interrupts on your timer before you are able to handle them, then it
> causes the timer to just stop.
>
> The REAL solution is to rewrite the IRQ_ENTRY and IRQ_EXIT to allow for
> re-entrance of IRQs (and not disable IRQs).

Interrupts are disabled automatically during interrupt service.
You need to reenable them after saving old register context and vice
versa at the end.
Interrupt routines have a register set of their own, but since you
can interrupt another interrupt you have to take care about it's
context.

> Our temporary solution involves using a wrapping timer instead of a
> resetting timer.  Additionally, if the timer's accuracy is highly
> important, you should check for overflow and handle it (ie: two
> interrupts before handler, should be detectable by amount in TC1_CV)

I fail to see how a missed interrupt can lead to timer stoping.
This sounds like a design failure somewhere else.
Reenabling interrupts adds more complexity, because interrupt routines
are not safe to modify global tables anymore - e.g. for posting an
event.
Of course loosing a timer interrupt is bad for acuracy.
Are there really any long running interrupt routines?
If there are this should be fixed instead.

--
B.Walter <bernd@...> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Coleman Brumley-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


> I fail to see how a missed interrupt can lead to timer stopping.

What was really peculiar was not only did the TC1 interrupt stop, but also
the TC0 interrupt which is the main Nut/OS timer.  This, of course, was
REALLY bad and caused all sorts of other cascading failures (threads failing
to execute, etc.).  

> This sounds like a design failure somewhere else.

The version posted by Timothy works as expected, i.e. the timer interrupt
doesn't stop working.  This version has now been running for almost 48 hours
without fail, whereas the sample posted at
http://www.ethernut.de/en/documents/at91-timer-irq.html, which is what my
code was using fails usually after a couple of hours.  

The only differences between the two register functions are:

-/* Select divider and compare trigger */
-outr(TC1_CMR, TC_CLKS_MCK32 | TC_CPCTRG);

+/* Select divider and compare trigger */
+outr(TC1_CMR, TC_CLKS_MCK32);

-/* set to lowest priority (0) */
-NutIrqSetPriority(&sig_TC1, 0);

+/* set to highest priority (7) */
+NutIrqSetPriority(&sig_TC1, 7);

And, there's the addition of the code to increment the compare register in
the handler:

+//increase compare value to next interrupt value
+//(signed short allows for overflow wrapping)
+#ifndef NUT_TICK_FREQ
+#define NUT_TICK_FREQ   1000UL
+#endif
+#if defined(AT91_PLL_MAINCK)
+ #define COMPARE_ADDER (At91GetMasterClock() / (32 * NUT_TICK_FREQ))

+#else
+    #define COMPARE_ADDER (NutGetCpuClock() / (32 * NUT_TICK_FREQ))
+#endif
+ compare_value += COMPARE_ADDER;
+ outr(TC1_RC, compare_value); //new compare value

> Reenabling interrupts adds more complexity, because interrupt routines
> are not safe to modify global tables anymore - e.g. for posting an
> event.

It's seems like this bug is in IRQ_ENTRY and IRQ_EXIT for the ARM7.  I'm not
an ARM assembly expert by any means, so this was out of my realm of comfort
for trying to address.  




_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Bernd Walter-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sun, Jul 12, 2009 at 11:37:55AM -0400, Coleman Brumley wrote:
>
> > I fail to see how a missed interrupt can lead to timer stopping.
>
> What was really peculiar was not only did the TC1 interrupt stop, but also
> the TC0 interrupt which is the main Nut/OS timer.  This, of course, was
> REALLY bad and caused all sorts of other cascading failures (threads failing
> to execute, etc.).  

Strange.

> > This sounds like a design failure somewhere else.
>
> The version posted by Timothy works as expected, i.e. the timer interrupt
> doesn't stop working.  This version has now been running for almost 48 hours
> without fail, whereas the sample posted at
> http://www.ethernut.de/en/documents/at91-timer-irq.html, which is what my
> code was using fails usually after a couple of hours.  
>
> The only differences between the two register functions are:
>
> -/* Select divider and compare trigger */
> -outr(TC1_CMR, TC_CLKS_MCK32 | TC_CPCTRG);
>
> +/* Select divider and compare trigger */
> +outr(TC1_CMR, TC_CLKS_MCK32);
>
> -/* set to lowest priority (0) */
> -NutIrqSetPriority(&sig_TC1, 0);
>
> +/* set to highest priority (7) */
> +NutIrqSetPriority(&sig_TC1, 7);
>
> And, there's the addition of the code to increment the compare register in
> the handler:
>
> +//increase compare value to next interrupt value
> +//(signed short allows for overflow wrapping)
> +#ifndef NUT_TICK_FREQ
> +#define NUT_TICK_FREQ   1000UL
> +#endif
> +#if defined(AT91_PLL_MAINCK)
> + #define COMPARE_ADDER (At91GetMasterClock() / (32 * NUT_TICK_FREQ))
>
> +#else
> +    #define COMPARE_ADDER (NutGetCpuClock() / (32 * NUT_TICK_FREQ))
> +#endif
> + compare_value += COMPARE_ADDER;
> + outr(TC1_RC, compare_value); //new compare value
>
> > Reenabling interrupts adds more complexity, because interrupt routines
> > are not safe to modify global tables anymore - e.g. for posting an
> > event.
>
> It's seems like this bug is in IRQ_ENTRY and IRQ_EXIT for the ARM7.  I'm not
> an ARM assembly expert by any means, so this was out of my realm of comfort
> for trying to address.  

There is nothing unusual.
It just does the normal things.
Save/restore some values and notify the AIC about processing.
I wonder if NutRegisterIrqHandler() registers level or edge.
I personally don't use it in my code and always register level, unless I
have special external hardware to care about.
Edge interrupts are always sensible to handle.
On the other hand: TC0 is a different vector than TC1, so it shouldn't
stop if you lock out TC1 interrupt.

But I never used an ARM TC for interrupts.
I've only used TC to generate external clocks.
I use a timer for RS485 on AVR, but since the ARM USART can handle
timeouts themself I had no use for my own timer interrupt anymore.
So far all interrupt rpbolems turned out to be my fault.

--
B.Walter <bernd@...> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Tim DeBaillie :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Bernd Walter wrote:

> Interrupts are disabled automatically during interrupt service.
> You need to reenable them after saving old register context and vice
> versa at the end.
> Interrupt routines have a register set of their own, but since you
> can interrupt another interrupt you have to take care about it's
> context.
>
>  
>> Our temporary solution involves using a wrapping timer instead of a
>> resetting timer.  Additionally, if the timer's accuracy is highly
>> important, you should check for overflow and handle it (ie: two
>> interrupts before handler, should be detectable by amount in TC1_CV)
>>    
>
> I fail to see how a missed interrupt can lead to timer stoping.
>  
It has been a while since I looked at it, but basically what happens is
the OS stops interrupting because the second interrupt is never queued
and therefore never cleared. Subsequent interrupts on the timer then are
missed. The timer is still running, it just doesn't cause an interrupt
in the OS. I have seen this on a logic analyzer and read out the CV
register which continued to change.

The odd behavior would be TC0 stopping as well. I don't have an
explanation for that.

> This sounds like a design failure somewhere else.
>  
I agree. We resolved our design failure (other interrupt taking too
long) and additionally added the "fix" for the timer stopping just
because we don't want the timer to stop in production code.

Tim DeBaillie
_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion

Re: AT91SAM7X256 TC1 Timer being disabled

by Coleman Brumley-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> It has been a while since I looked at it, but basically what happens is
> the OS stops interrupting because the second interrupt is never queued
> and therefore never cleared. Subsequent interrupts on the timer then
> are
> missed. The timer is still running, it just doesn't cause an interrupt
> in the OS. I have seen this on a logic analyzer and read out the CV
> register which continued to change.
>
> The odd behavior would be TC0 stopping as well. I don't have an
> explanation for that.

I apologize for being inarticulate in my earlier postings.  In hindsight, my
post insinuated that the TC0 and TC1 _interrupts_ were stopping at the CPU
level. That's not accurate.  TC1 was stopping at the interrupt handler level
(this was fixed by Timothy's code), and I was never able to determine where
TC0 was stopping.  

What is happening is that the Nut/OS handling of the TC0 timer stops.  So,
the following code stops working.

timer5 = NutTimerStart(5, TimerCallback5, (void*)&event1, 0);

///////////////////////////////////////////////////////////////////////
//  raises an event every 5ms

void TimerCallback5(HANDLE timer, void* arg)
{
        NutEventPostAsync((HANDLE*)arg);
}

I had toggled one of my I/O pins in TimerCallback5 and captured it on a
scope and the I/O pin stopped toggling.  This would also cause NutEvenWait
to never timing out.  So, anything waiting on event1 would not run.  The
problem seemed worse when the system was under heavy UART traffic.

Putting 2 and 2 together (from Harald's recent post about Nut/OS tasks):

"Right now the most time consuming interrupts are in the UART drivers,
because buffering incoming characters or acting on hardware handshake needs
to be done in shortest time."

And Timothy's description of their design failure in that one of their
interrupt handlers was taking too long...

Perhaps there's something broken in the UART interrupt handler that's
causing a long delay and therefore causing this problem?

>
> > This sounds like a design failure somewhere else.
> >
> I agree. We resolved our design failure (other interrupt taking too
> long) and additionally added the "fix" for the timer stopping just
> because we don't want the timer to stop in production code.
>

I've also implemented Timothy's fix because if the timer stops in production
code, then the product stops working all together.  

Regards,
Coleman


_______________________________________________
http://lists.egnite.de/mailman/listinfo/en-nut-discussion