[jira] Created: (DERBY-4432) Memory leak when attempting to boot non-existing database with the in-memory back end

Memory leak when attempting to boot non-existing database with the in-memory back end
-------------------------------------------------------------------------------------

                 Key: DERBY-4432
                 URL: https://issues.apache.org/jira/browse/DERBY-4432
             Project: Derby
          Issue Type: Bug
          Components: Store
    Affects Versions: 10.5.3.0, 10.6.0.0
         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
            Reporter: Kristian Waagan
            Assignee: Kristian Waagan


When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).

FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Waagan updated DERBY-4432:
-----------------------------------

    Attachment: derby-4432-1a-mem_leak_fix.stat
                derby-4432-1a-mem_leak_fix.diff

Patch 1a fixes the memory leak and adds a test for it.

A "dummy" data store has been added. It will be used until Derby actually requests a database to be created (when a unique name is provided).
Directories for created databases will be created in the dummy data store, which means it will grow with the number of databases created. I don't think this will be a problem, because the space occupied by the databases themselves will be much larger than the bytes required to store a directory structure. Further, I intended to delete the directory structure in the dummy store for a database when the database itself is deleted (this patch will probably appear under DERBY-4428).

An alternative to the current patch is to use a bounded map where the oldest entry is deleted when the bound has been exceeded. I discarded this approach because the code got slightly more complex.

Patch ready for review.

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Waagan updated DERBY-4432:
-----------------------------------

    Issue & fix info: [Patch Available]

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773006#action_12773006 ]

Kristian Waagan commented on DERBY-4432:
----------------------------------------

Forgot to mention that patch 1a changes the behavior slightly. The in-memory back end will now behave more like the on-disk back end in that you won't be able to create database "/tmp/a" if you have already created database "/tmp/a/b" (there may be some exceptions for both back ends when Derby is instructed to delete the existing service root). The reason is the check for whether the service root already exists or not.

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775372#action_12775372 ]

Kristian Waagan commented on DERBY-4432:
----------------------------------------

Regression tests ran cleanly on trunk with patch 1a.
Committed patch 1a to trunk with revision 834418.

Will check if it is suited for back-port and rerun tests on the 10.5 branch if that is the case.

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>             Fix For: 10.6.0.0
>
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Waagan updated DERBY-4432:
-----------------------------------

    Fix Version/s: 10.6.0.0

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>             Fix For: 10.6.0.0
>
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Waagan updated DERBY-4432:
-----------------------------------

    Attachment: releaseNote.html

Back-ported patch 1a to the 10.5 branch with revision 835290.
Attached first revision of a release note, as the change may break existing applications using the in-memory back end. The incompatibility will normally not be triggered, which is why I chose to back-port the fix.

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>             Fix For: 10.6.0.0
>
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat, releaseNote.html
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kristian Waagan resolved DERBY-4432.
------------------------------------

          Resolution: Fixed
       Fix Version/s: 10.5.3.1
    Issue & fix info: [Release Note Needed]  (was: [Patch Available])

> Memory leak when attempting to boot non-existing database with the in-memory back end
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-4432
>                 URL: https://issues.apache.org/jira/browse/DERBY-4432
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.5.3.0, 10.6.0.0
>         Environment: In-memory back end and large amount of boot attempts of non-existing databases.
>            Reporter: Kristian Waagan
>            Assignee: Kristian Waagan
>             Fix For: 10.5.3.1, 10.6.0.0
>
>         Attachments: derby-4432-1a-mem_leak_fix.diff, derby-4432-1a-mem_leak_fix.stat, releaseNote.html
>
>
> When attempting to boot a non-existing database with the in-memory back end, a DataStore is created and referenced by a map. Since the database doesn't exist, Derby won't use the DataStore again and it will hang around indefinitely in VFMemoryStorageFactory (or until the same database is actually created, in which case the data store will be used).
> FYI, storage factory instances are instantiated liberally, including for the same service (i.e. six instances of VFMemoryStorageFactory may be created to boot "jdbc:derby:memory:myDB").

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.