Re: Repairing a corrupted database with invalid checksum on page

Kathey Marsden wrote:
> Mike Matrigali wrote:
>> Then you could figure out how important container 1105
>> is.  The best case is if 1105 is an index then one need only drop the
>> index and recreate it.  From the conglomerate id I think it is an
>> index:
>>
> Once you get your database in a bootable state you can identify the
> container with one of the following queries.

Hi Mike and Kathey,

Thank you very much for your helpful replies.  As suggested, I wrote a
hacked version of Derby to ignore this page during the log redo phase.

The database booted up, and thankfully, the corrupted page was in an
index, so I was able to drop it, and recreate it and now all is well.

For the record, I've attached the patch I made to Derby in case somebody
else can benefit from it.  In this situation, I ran Derby's ij command with:

java -cp derby.jar;derbytools.jar
-Dorg.apache.derby.skipPagesDuringRecovery=3586

The system property specifies a comma separated string of page IDs to
ignore during the log redo phase.

I realise this is hackery, but would there be any benefit for a polished
version of this to go into the main code?  Sadly, corruption is a fact
of life, whether it is due to software or hardware (or humans)..

--
Cheers,
David

Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia    Ph: +61 2 9280 0699
Web: http://www.nuix.com                            Fax: +61 2 9212 6902

Index: dist/java/engine/org/apache/derby/impl/store/raw/data/PageBasicOperation.java
===================================================================
--- dist/java/engine/org/apache/derby/impl/store/raw/data/PageBasicOperation.java (revision 6493)
+++ dist/java/engine/org/apache/derby/impl/store/raw/data/PageBasicOperation.java Thu Jun 11 16:33:52 EST 2009
@@ -52,7 +52,11 @@
 import java.io.ObjectOutput;
 import java.io.ObjectInput;
 import java.io.IOException;
+import java.util.Set;
+import java.util.HashSet;
+
 import org.apache.derby.iapi.services.io.LimitObjectInput;
+import org.apache.derby.iapi.services.monitor.Monitor;
 
 /**
     A PageBasicOperation changed the content of a page, this is the root class of all
@@ -157,6 +161,28 @@
  public final boolean needsRedo(Transaction xact)
  throws StandardException
  {
+        // Skip certain pages during recovery if we have corrupted pages as a
+        // way of trying to get a corrupted database into a useable state.
+        if (pageId != null)
+        {
+            String skipPagesProperty = System.getProperty("org.apache.derby.skipPagesDuringRecovery");
+            Set skipPages = new HashSet();
+            if (skipPagesProperty != null)
+            {
+                String[] skipPagesArray = skipPagesProperty.split(",");
+                for (int i = 0; i < skipPagesArray.length; i++)
+                {
+                    skipPages.add(new Long(Long.parseLong(skipPagesArray[i])));
+                }
+            }
+
+            if (skipPages.contains(new Long(pageId.getPageNumber())))
+            {
+                Monitor.getStream().println("Skipping page " + pageId.getPageNumber() + " during log redo");
+                return false;
+            }
+        }
+        
  if (findpage(xact) == null) // committed dropped container
  return false;