[GSoC] Automated Testing for Improved dblook Implementation

[Hiranya Jayathilaka-3]

Hi Folks,

Now that I've reached the final stages of my project to improve dblook I think it's time I start thinking about writing some automated tests for the new and improved dblook implementation. Dag came up with the idea of getting dblook to generate a script against some reference database and then run the script to create a copy of the reference database. Then the copy will be compared with the reference database to verify that all persistent objects like tables and views are properly in place. Eventhough it sounds like lot of work it's certainly doable and I think this is a great way to test dblook. We will probably have to modify the script generated by dblook by filling in the passwords. That should also take place in the automated environment.

I would like to know what the community thinks about this testing strategy. Also I would appreciate if you can provide your own ideas on this issue.


Thanks,
Hiranya Jayathilaka

[Rick Hillegas-2]

Hiranya Jayathilaka wrote:

> Hi Folks,
>
> Now that I've reached the final stages of my project to improve dblook
> I think it's time I start thinking about writing some automated tests
> for the new and improved dblook implementation. Dag came up with the
> idea of getting dblook to generate a script against some reference
> database and then run the script to create a copy of the reference
> database. Then the copy will be compared with the reference database
> to verify that all persistent objects like tables and views are
> properly in place. Eventhough it sounds like lot of work it's
> certainly doable and I think this is a great way to test dblook. We
> will probably have to modify the script generated by dblook by filling
> in the passwords. That should also take place in the automated
> environment.
>
> I would like to know what the community thinks about this testing
> strategy. Also I would appreciate if you can provide your own ideas on
> this issue.

Hi Hiranya,

I think this sounds like a reasonable approach. The recently written
UpgradeTrajectoryTest does something similar: it compares the metadata
in Before and After databases. You may get some implementation ideas if
you glance at that test. Two tricky bits about comparing metadata are 1)
you need to remap UUIDs, 2) timestamp columns are unstable.
UpgradeTrajectoryTest.normalizeRow() may help you sort through the
expected differences.

Hope this helps,
-Rick
>
>
> Thanks,
> Hiranya Jayathilaka
>

[Dag H. Wanvik]

Rick Hillegas <Richard.Hillegas@...> writes:

> Hiranya Jayathilaka wrote:
>> Hi Folks,
>>
>> Now that I've reached the final stages of my project to improve
>> dblook I think it's time I start thinking about writing some
>> automated tests for the new and improved dblook implementation. Dag
>> came up with the idea of getting dblook to generate a script against
>> some reference database and then run the script to create a copy of
>> the reference database. Then the copy will be compared with the
>> reference database to verify that all persistent objects like tables
>> and views are properly in place. Eventhough it sounds like lot of

You would also want to check that the permissions and role tables are
the same.

[Note that the dependency tables might be hard to compare: For example
if a view was constructed on the virtue of having a role set, that
view would be dependent on the role in the original database. Later, a
direct permission to the user in question may have been added. Then,
in a reconstructed database, a direct user permission is "preferred"
(checked first) when the view is reconstructed and, hence, the
dependencies would be different (although otherwise the databases
schemas would be the same, and allowed operations would be the
same). Such subtle differences would only make a difference when
revoking privileges or roles. Ideally the order in which privileges
are given should not matter; it only does matter due to DERBY-1632. I
think you should ignore this effect for now. :) It may merit a
footnote in the (updated) docs for dblook.. ]

Dag