Filesystem Testing

[Paolo Ambrosio]

Hello,

I am starting my first TDD project (multiplatform C++) and I ran into
some issues regarding how to test it. The system is based on
filesystem interations (it should read the filesystem and handle file
change events), so I'd like that component to be heavily tested. How
should I do that? I am going to...

 * Abstract the filesystem interface (I need it anyway to handle both
Windows and Unix)
 * Write a test double implementing that interface
 * Run the unit tests on the other components of the system using the
test double
 * Run functional integration tests on the real filesystem abstraction
classes (those using the OS APIs)

Is this overkill? Is there another way I can do TDD on this system?
Did I forget something? Any advice is appreciated.


Thanks,
Paolo

[Carlos Ble-2]

Hi Paolo,
It sounds good to me, just remember writing the tests first :-)

Cheers

2009/8/30 Paolo Ambrosio <paolo.ambrosio@...>:

>
>
> Hello,
>
> I am starting my first TDD project (multiplatform C++) and I ran into
> some issues regarding how to test it. The system is based on
> filesystem interations (it should read the filesystem and handle file
> change events), so I'd like that component to be heavily tested. How
> should I do that? I am going to...
>
> * Abstract the filesystem interface (I need it anyway to handle both
> Windows and Unix)
> * Write a test double implementing that interface
> * Run the unit tests on the other components of the system using the
> test double
> * Run functional integration tests on the real filesystem abstraction
> classes (those using the OS APIs)
>
> Is this overkill? Is there another way I can do TDD on this system?
> Did I forget something? Any advice is appreciated.
>
> Thanks,
> Paolo
>
>

[Mark Levison-2]

I do hard stuff like test doubles only as needed.

So I would:
- start test first
- abstract away the filesystem when I did something that forced me to port
- implement test doubles for the filesystem only when things become too slow
- write some separate end to end tests (aka acceptance tests) that exercise
everything including the file system.

Cheers
Mark

On Mon, Aug 31, 2009 at 8:19 AM, Carlos Ble <ble.jurado@...> wrote:

>
>
> Hi Paolo,
> It sounds good to me, just remember writing the tests first :-)
>
> Cheers
>
> 2009/8/30 Paolo Ambrosio <paolo.ambrosio@...<paolo.ambrosio%40gmail.com>
> >:
>
> >
> >
> > Hello,
> >
> > I am starting my first TDD project (multiplatform C++) and I ran into
> > some issues regarding how to test it. The system is based on
> > filesystem interations (it should read the filesystem and handle file
> > change events), so I'd like that component to be heavily tested. How
> > should I do that? I am going to...
> >
> > * Abstract the filesystem interface (I need it anyway to handle both
> > Windows and Unix)
> > * Write a test double implementing that interface
> > * Run the unit tests on the other components of the system using the
> > test double
> > * Run functional integration tests on the real filesystem abstraction
> > classes (those using the OS APIs)
> >
> > Is this overkill? Is there another way I can do TDD on this system?
> > Did I forget something? Any advice is appreciated.
> >
> > Thanks,
> > Paolo
> >
> >
>  
>

   *Mark Levison* | Founder and Consultant -
TheAgileConsortium<http://theagileconsortium.com/>| Agile
Editor @ InfoQ <http://www.infoq.com/about.jsp>
Blog <http://www.notesfromatooluser.com/> |
Twitter<http://twitter.com/mlevison>| Office: [phone]
Recent Entries: Agile Mailing Lists a
Survey<http://www.notesfromatooluser.com/2009/06/agile-mailing-lists.html%20>,
Do You Suspect You Have a Less than Productive Person on Your
Team?<http://www.notesfromatooluser.com/2009/01/do-you-suspect-you-have-a-less-than-productive-person-on-your-team.html>


[Non-text portions of this message have been removed]

[franz see]

IMHO, you should write tests for your business logic first (abstracted from
the filesystem via test doubles).

But once you start touching the file system (i.e. with your
FileSystemManager or your FileSystemUtility or whatever), I suggest you test
with actual files.

Though you can 'abstract' your application from the actual file system, you
are running the risk of not having actual scenarios being ran. It is only
when you interact with the file systems that you notice scenarios that you
did not realize before.

Sure, it's an integration test - but that is what you need anyway. Sure it
may be slower - but compared to restarting your application for you to
handle another scenario which your 'mocked' scenarios did not cover, it's
faster.

Btw, there's another recent thread in this mailing regarding file system.
Please feel free to take a look at it :-)

Cheers,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@...
LinkedIn: http://www.linkedin.com/in/franzsee

On Sun, Aug 30, 2009 at 10:45 PM, Paolo Ambrosio
<paolo.ambrosio@...>wrote:

>
>
> Hello,
>
> I am starting my first TDD project (multiplatform C++) and I ran into
> some issues regarding how to test it. The system is based on
> filesystem interations (it should read the filesystem and handle file
> change events), so I'd like that component to be heavily tested. How
> should I do that? I am going to...
>
> * Abstract the filesystem interface (I need it anyway to handle both
> Windows and Unix)
> * Write a test double implementing that interface
> * Run the unit tests on the other components of the system using the
> test double
> * Run functional integration tests on the real filesystem abstraction
> classes (those using the OS APIs)
>
> Is this overkill? Is there another way I can do TDD on this system?
> Did I forget something? Any advice is appreciated.
>
> Thanks,
> Paolo
>  
>

[Non-text portions of this message have been removed]

[darose]

On 09/04/2009 07:39 AM, Franz Allan Valencia See wrote:
> But once you start touching the file system (i.e. with your
> FileSystemManager or your FileSystemUtility or whatever), I suggest you test
> with actual files.
>
> Though you can 'abstract' your application from the actual file system, you
> are running the risk of not having actual scenarios being ran. It is only
> when you interact with the file systems that you notice scenarios that you
> did not realize before.

I'd agree.  Mocking all of the behavior of a file system gets very
involved after a while, and you wind up spending a lot of time
developing code that's completely peripheral to the core functions of
your application.  I tried going that route in the early days of my TDD,
but quickly discovered that it was *much* easier to just create a
directory on /tmp and do your tests there.

Although I do agree with the premise that unit tests should not be
dependent on external resources (e.g., a database) I would not count a
local file system in that category.  99.9% of machines have a fast local
disk attached to it, and so I don't see any problem with writing unit
tests that rely on that fact.

DR

[Steven Smith-23]

Having done it both ways, I much prefer mocking the file system.  The
main issue in our case was portability of the dev environment between
various developers' machines, a build server, VPCs, etc.  We still
have integration tests that use the real file system in a stable setup
(build server) but the must-run-before-every-checkin unit tests all
work against an IFileSystem interface that makes it very easy to
decouple the actual file system operations from the business tasks
being performed.

YMMV.

Steve


On Fri, Sep 4, 2009 at 10:58 AM, David Rosenstrauch<darose@...> wrote:

>
>
> On 09/04/2009 07:39 AM, Franz Allan Valencia See wrote:
>> But once you start touching the file system (i.e. with your
>> FileSystemManager or your FileSystemUtility or whatever), I suggest you
>> test
>> with actual files.
>>
>> Though you can 'abstract' your application from the actual file system,
>> you
>> are running the risk of not having actual scenarios being ran. It is only
>> when you interact with the file systems that you notice scenarios that you
>> did not realize before.
>
> I'd agree. Mocking all of the behavior of a file system gets very
> involved after a while, and you wind up spending a lot of time
> developing code that's completely peripheral to the core functions of
> your application. I tried going that route in the early days of my TDD,
> but quickly discovered that it was *much* easier to just create a
> directory on /tmp and do your tests there.
>
> Although I do agree with the premise that unit tests should not be
> dependent on external resources (e.g., a database) I would not count a
> local file system in that category. 99.9% of machines have a fast local
> disk attached to it, and so I don't see any problem with writing unit
> tests that rely on that fact.
>
> DR
>
>

--
Steve Smith
http://SteveSmithBlog.com/
http://twitter.com/ardalis

[Donaldson, John (GEO)]

Another factor which might influence how you test would be your focus. You may be really focused on the file system - manipulating files. It may be a central part of your app. On the other hand, you may be interested in files just as a place to get some persistence. In which case, you might be better off having a streaming abstraction of some kind. With one implementation of the end of a stream being a file.

John D.

-----Original Message-----
From: testdrivendevelopment@... [mailto:testdrivendevelopment@...] On Behalf Of Steven Smith
Sent: 04 September 2009 17:05
To: testdrivendevelopment@...
Subject: Re: [TDD] Filesystem Testing

Having done it both ways, I much prefer mocking the file system.  The
main issue in our case was portability of the dev environment between
various developers' machines, a build server, VPCs, etc.  We still
have integration tests that use the real file system in a stable setup
(build server) but the must-run-before-every-checkin unit tests all
work against an IFileSystem interface that makes it very easy to
decouple the actual file system operations from the business tasks
being performed.

YMMV.

Steve


On Fri, Sep 4, 2009 at 10:58 AM, David Rosenstrauch<darose@...> wrote:
>
>
> On 09/04/2009 07:39 AM, Franz Allan Valencia See wrote:
>> But once you start touching the file system (i.e. with your
>> FileSystemManager or your FileSystemUtility or whatever), I suggest you
>> test
>> with actual files.

[franz see]

I agree. You should abstract your business logic from the infrastructure.
But when testing the actual file system operations (i.e. your
FileSystemManager, etc), mocking them might just give you false hope that
everything's ok.

Cheers,

--
Franz Allan Valencia See | Java Software Engineer
franz.see@...
LinkedIn: http://www.linkedin.com/in/franzsee

On Fri, Sep 4, 2009 at 11:33 PM, Donaldson, John (GEO) <
john.m.donaldson@...> wrote:

>
>
> Another factor which might influence how you test would be your focus. You
> may be really focused on the file system - manipulating files. It may be a
> central part of your app. On the other hand, you may be interested in files
> just as a place to get some persistence. In which case, you might be better
> off having a streaming abstraction of some kind. With one implementation of
> the end of a stream being a file.
>
> John D.
>
>
> -----Original Message-----
> From: testdrivendevelopment@...<testdrivendevelopment%40yahoogroups.com>[mailto:
> testdrivendevelopment@...<testdrivendevelopment%40yahoogroups.com>]
> On Behalf Of Steven Smith
> Sent: 04 September 2009 17:05
> To: testdrivendevelopment@...<testdrivendevelopment%40yahoogroups.com>
> Subject: Re: [TDD] Filesystem Testing
>
> Having done it both ways, I much prefer mocking the file system. The
> main issue in our case was portability of the dev environment between
> various developers' machines, a build server, VPCs, etc. We still
> have integration tests that use the real file system in a stable setup
> (build server) but the must-run-before-every-checkin unit tests all
> work against an IFileSystem interface that makes it very easy to
> decouple the actual file system operations from the business tasks
> being performed.
>
> YMMV.
>
> Steve
>
> On Fri, Sep 4, 2009 at 10:58 AM, David Rosenstrauch<darose@...<darose%40darose.net>>
> wrote:
> >
> >
> > On 09/04/2009 07:39 AM, Franz Allan Valencia See wrote:
> >> But once you start touching the file system (i.e. with your
> >> FileSystemManager or your FileSystemUtility or whatever), I suggest you
> >> test
> >> with actual files.
>  
>

--
Franz Allan Valencia See | Java Software Engineer
franz.see@...
LinkedIn: http://www.linkedin.com/in/franzsee


[Non-text portions of this message have been removed]

[darose]

Yes, I do this often as well - i.e., program to an input or output
stream instead of to a file.  Makes it very easy to substitute in a
memory-based stream for unit testing.

DR

On 09/04/2009 11:33 AM, Donaldson, John (GEO) wrote:

> Another factor which might influence how you test would be your
> focus. You may be really focused on the file system - manipulating
> files. It may be a central part of your app. On the other hand, you
> may be interested in files just as a place to get some persistence.
> In which case, you might be better off having a streaming abstraction
> of some kind. With one implementation of the end of a stream being a
> file.
>
> John D.
>
> -----Original Message----- From:
> testdrivendevelopment@...
> [mailto:testdrivendevelopment@...] On Behalf Of Steven
> Smith Sent: 04 September 2009 17:05 To:
> testdrivendevelopment@... Subject: Re: [TDD] Filesystem
> Testing
>
> Having done it both ways, I much prefer mocking the file system.  The
>  main issue in our case was portability of the dev environment
> between various developers' machines, a build server, VPCs, etc.  We
> still have integration tests that use the real file system in a
> stable setup (build server) but the must-run-before-every-checkin
> unit tests all work against an IFileSystem interface that makes it
> very easy to decouple the actual file system operations from the
> business tasks being performed.
>
> YMMV.
>
> Steve
>
>
> On Fri, Sep 4, 2009 at 10:58 AM, David
> Rosenstrauch<darose@...> wrote:
>>
>> On 09/04/2009 07:39 AM, Franz Allan Valencia See wrote:
>>> But once you start touching the file system (i.e. with your
>>> FileSystemManager or your FileSystemUtility or whatever), I
>>> suggest you test with actual files.
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>