[Bug 14256] New: kernel BUG at fs/ext3/super.c:435

http://bugzilla.kernel.org/show_bug.cgi?id=14256

           Summary: kernel BUG at fs/ext3/super.c:435
           Product: File System
           Version: 2.5
    Kernel Version: 2.6.31
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: ext3
        AssignedTo: fs_ext3@...
        ReportedBy: rjw@...
            Blocks: 13615
        Regression: Yes


Subject    : [2.6.31] kernel BUG at fs/ext3/super.c:435
Submitter  : Mikael Pettersson <mikpe@...>
Date       : 2009-09-21 7:29
References : http://marc.info/?l=linux-kernel&m=125351816109264&w=4

This entry is being used for tracking a regression from 2.6.30.  Please don't
close it until the problem is fixed in the mainline.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #1 from Rafael J. Wysocki <rjw@...>  2009-10-04 20:47:41 ---
On Sunday 04 October 2009, Mikael Pettersson wrote:

> Rafael J. Wysocki wrote:
> > The following bug entry is on the current list of known regressions
> > introduced between 2.6.30 and 2.6.31.  Please verify if it still should
> > be listed and let me know (either way).
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=3D14256
> > Subject : kernel BUG at fs/ext3/super.c:435
> > Submitter : Mikael Pettersson <mikpe@...>
> > Date : 2009-09-21 7:29 (11 days old)
> > References : http://marc.info/?l=3Dlinux-kernel&m=3D125351816109264&w=3D4
>
> The exact same bug (same cause, same symptom) just hit me again in 2.6.32-rc1.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #2 from Rafael J. Wysocki <rjw@...>  2009-10-09 22:01:43 ---
On Friday 09 October 2009, Mikael Pettersson wrote:

> Mikael Pettersson writes:
>  > Rafael J. Wysocki writes:
>  >  > On Sunday 04 October 2009, Mikael Pettersson wrote:
>  >  > > Rafael J. Wysocki wrote:
>  >  > > > The following bug entry is on the current list of known regressions
>  >  > > > introduced between 2.6.30 and 2.6.31.  Please verify if it still should
>  >  > > > be listed and let me know (either way).
>  >  > > >
>  >  > > >
>  >  > > > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=3D14256
>  >  > > > Subject : kernel BUG at fs/ext3/super.c:435
>  >  > > > Submitter : Mikael Pettersson <mikpe@...>
>  >  > > > Date : 2009-09-21 7:29 (11 days old)
>  >  > > > References : http://marc.info/?l=3Dlinux-kernel&m=3D125351816109264&w=3D4
>  >  > >
>  >  > > The exact same bug (same cause, same symptom) just hit me again in 2.6.32-rc1.
>  >  >
>  >  > Thanks for the update.
>  >  >
>  >  > Could you check the current Linus' tree, please?  There are some known
>  >  > regression fixes in there.
>  >
>  > I tried simplified versions of the bug trigger on two machines
>  > running 2.6.32-rc1-git6, and neither triggered the kernel bug.
>  >
>  > The original recipe involved doing a glibc rebuild, run its test
>  > suite, install it, and reboot. Today however machine 1 was already
>  > doing a rebuild so after the rebuild it did a reboot into the new
>  > kernel before the install. The second machine booted the new kernel
>  > directly to install the binary packages from the first machine.
>  >
>  > I'll re-run the full bug trigger recipe on a third machine later next
>  > week (it must rebuild glibc itself anyway due to arch differences).
>
> Not fixed in 2.6.32-rc3. A glibc rebuild + install triggered the
> exact same bug on the third machine.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Theodore Ts'o-2]

On Fri, Oct 09, 2009 at 10:01:44PM +0000, bugzilla-daemon@... wrote:
> > Not fixed in 2.6.32-rc3. A glibc rebuild + install triggered the
> > exact same bug on the third machine.

Can you isolate this down to something simpler?  Will "make install"
followed by a reboot be sufficient to replicate the assertion failure?
If it does, will copying in a new version of libc.so.6 and then
rebooting be enough to trigger the assert failure?

Thanks,

                             - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #3 from Theodore Tso <tytso@...>  2009-10-10 01:31:41 ---
On Fri, Oct 09, 2009 at 10:01:44PM +0000, bugzilla-daemon@...
wrote:
> > Not fixed in 2.6.32-rc3. A glibc rebuild + install triggered the
> > exact same bug on the third machine.

Can you isolate this down to something simpler?  Will "make install"
followed by a reboot be sufficient to replicate the assertion failure?
If it does, will copying in a new version of libc.so.6 and then
rebooting be enough to trigger the assert failure?

Thanks,

                                            - Ted

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Mikael Pettersson-3]

Theodore Tso writes:
 > On Fri, Oct 09, 2009 at 10:01:44PM +0000, bugzilla-daemon@... wrote:
 > > > Not fixed in 2.6.32-rc3. A glibc rebuild + install triggered the
 > > > exact same bug on the third machine.
 >
 > Can you isolate this down to something simpler?  Will "make install"
 > followed by a reboot be sufficient to replicate the assertion failure?

I've done repeated glibc downgrades and upgrades, using pre-built binary
packages, followed by reboots, but they have so far not triggered the bug.

My guess is that something in the build process or the test suite run
triggers the bug.

FWIW, the systems are rpm-based derived from Fedora, so the rebuilds
are just "rpmbuild --rebuild glibc-<version>.src.rpm", and the installs
are "rpm -Uvh" on the binary rpms, with --force if I'm doing a downgrade.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #4 from Mikael Pettersson <mikpe@...>  2009-10-12 13:48:39 ---
Theodore Tso writes:
 > On Fri, Oct 09, 2009 at 10:01:44PM +0000,
bugzilla-daemon@... wrote:
 > > > Not fixed in 2.6.32-rc3. A glibc rebuild + install triggered the
 > > > exact same bug on the third machine.
 >
 > Can you isolate this down to something simpler?  Will "make install"
 > followed by a reboot be sufficient to replicate the assertion failure?

I've done repeated glibc downgrades and upgrades, using pre-built binary
packages, followed by reboots, but they have so far not triggered the bug.

My guess is that something in the build process or the test suite run
triggers the bug.

FWIW, the systems are rpm-based derived from Fedora, so the rebuilds
are just "rpmbuild --rebuild glibc-<version>.src.rpm", and the installs
are "rpm -Uvh" on the binary rpms, with --force if I'm doing a downgrade.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256


Eric Sandeen <sandeen@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sandeen@...




--- Comment #5 from Eric Sandeen <sandeen@...>  2009-10-12 16:19:54 ---
I can give this a whirl on fedora.  Just to try to reproduce more exactly,
which version of the glibc rpm package are you rebuilding?

I'm also curious, does simply rebuilding the package and rebooting lead to the
same issue (w/o installing the result?)

Do you have rpm configured in any unique way or is the build happening in all
the normal/standard places?

So you have /tmp as ext3 as a separate fs, any other interesting fs config?

Thanks,
-Eric

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256


Jan Kara <jack@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jack@...




--- Comment #6 from Jan Kara <jack@...>  2009-11-05 16:07:31 ---
Eric, have you succeeded in reproducing this?

What seems to happen is that while we are in ext3_put_super(), there are still
some inode references held and therefore the inodes (regular files) don't get
deleted and we complain about them in ext3_put_super. The question is who holds
the reference... I did a quick audit of the code and couldn't find iget()
unmatched with iput() or a similar problem.

I guess I can create a debugging patch which might be able to show us who holds
the reference. Will try to code it next week. Mikael, will you be able to run
such debugging patch?

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #7 from Eric Sandeen <sandeen@...>  2009-11-05 16:13:15 ---
Jan, nope haven't, to be honest fell off my radar as I was looking at that ext4
corruption.  I've not tried to reproduce it yet.

Sorry,
-Eric

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #8 from Jan Kara <jack@...>  2009-11-11 15:27:53 ---
Created an attachment (id=23749)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=23749)
Patch reporting lost inode references

This patch reports more information about inodes left in orphan list.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #9 from Jan Kara <jack@...>  2009-11-11 15:29:18 ---
Mikael, could you please run a kernel with the above patch and post here
resulting messages during umount and also attach here your System.map file from
the tested kernel? Thanks.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256


Theodore Tso <tytso@...> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tytso@...




--- Comment #10 from Theodore Tso <tytso@...>  2009-11-20 15:11:21 ---
Hi Mikael,

Have you had a chance to try running a kernel with Jan's debugging patch?  

Is this a problem you can reproduce reliably, or was it a one-off?

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html

http://bugzilla.kernel.org/show_bug.cgi?id=14256





--- Comment #11 from Rafael J. Wysocki <rjw@...>  2009-11-21 14:27:39 ---
On Saturday 21 November 2009, Mikael Pettersson wrote:

> Rafael J. Wysocki wrote:
> > The following bug entry is on the current list of known regressions
> > introduced between 2.6.30 and 2.6.31.  Please verify if it still should
> > be listed and let me know (either way).
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=3D14256
> > Subject : kernel BUG at fs/ext3/super.c:435
> > Submitter : Mikael Pettersson <mikpe@...>
> > Date : 2009-09-21 7:29 (57 days old)
> > References : http://marc.info/?l=3Dlinux-kernel&m=3D125351816109264&w=3D4
>
> Still occurs with kernel 2.6.32-rc8.
>
> This time I only rebuilt the glibc src rpm, which also runs its
> test suite, but I did not install the new glibc binaries before
> the shutdown -r.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html