[PATCH] Added CONFIG_VFAT_FS_DUALNAMES option

[Pavel Machek]

> > Ultimately, though, requiring that every single possible device be
> > tested is probably not reasonable, so the best way to do this testing
> > is the way do most of our testing; we do basic due diligence, but then
> > we merge it into mainline and let our huge user community try it out.
> > If there are regressions we can work through those issues if and when
> > they arise.
>
> From the funnies we've had in the past with FAT my gut impression is
> there are only a few implementations out there. Psion seems to have their
> own but most of the rest behave remarkably similarly which makes me
> suspect they all licensed a tiny number of implementations (DRDOS one
> perhaps ?). If we can keep most of those devices mounted 8.3 we nicely
> sidestep the issue anyway.

I'm pretty sure there's more. There's stuff such as 'make your own mp3
player with pic and few more circuits'. I'd actually expect many mp3
players to be affected by this; many are really cheap & nasty.

                                                                Pavel

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Pavel Machek]

Hi!

>  > Defaults should be back-compatible.
>
> I would usually agree, but I think we have an unusual situation here,
> in some ways similar to a demonstrated security hole. The previous
> behaviour exposed a lot of Linux vendors to the possibility of an
> expensive legal fight.

I'd actually like to see all-out software patent war in the U.S. It
would make sure that software patents do not spread to the rest of
world. Bad for U.S.? Yes. Good for world? Yes!

>  > Users considering disabling this should understand that filesystem
>  > they write to will not be valid vfat filesystems, and may trigger bugs
>  > in some devices.
>
> If we find any devices that exhibit any problems with this patch while
> it is in linux-next (and maybe linux-mm?) then this warning would
> indeed be appropriate. It no such devices are known then I think the
> warning is going a bit far.

You already know that it breaks XP and older linuxes. So... what are
you arguing about?! Chkdsk marks it as invalid filesystem... not a
vfat.

If mickey$oft intentionally modified windows 8 to intentionally corrupt
ext3 filesystems so that linux oopses on accessing them, how'd you
like that?

>  > Why not use something like position in directory instead of random
>  > number?
>
> We did of course consider that, and the changes to the patch to
> implement collision avoidance are relatively simple. We didn't do it
> as it would weaken the legal basis behind the patch. I'll leave it to

Consider it again. Or put fair 'this makes XP crash' warning in
kconfig.

                                                                Pavel
PS: I find it bad that original patch description did not contain XP
crashing / chkdsk explanation.
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Pavel Machek]

On Wed 2009-07-01 10:44:47, James Bottomley wrote:

> On Wed, 2009-07-01 at 14:48 +0300, Boaz Harrosh wrote:
> > On 07/01/2009 01:50 PM, tridge@... wrote:
> > > Hi Pavel,
> > >
> > > We did of course consider that, and the changes to the patch to
> > > implement collision avoidance are relatively simple. We didn't do it
> > > as it would weaken the legal basis behind the patch. I'll leave it to
> > > John Lanza (the LF patent attorney) to expand on that if you want more
> > > information.
> > >
> >
> > You completely lost me here. And I thought I did understand the patent
> > and the fix.
> >
> > what is the difference between.
> >
> > short_name = rand(sid);
> > and
> > short_name = sid++;
> >
> > Now if you would do
> > short_name = MD5(long_name);
> >
> > That I understand since short_name is some function of long_name
> > but if I'm just inventing the short_name out of my hat. In what legal
> > system does it matter what is my random function I use?
>
> We're sort of arguing moot technicalities here.  If you look at the way
> the filename is constructed, given the constraints of a leading space
> and a NULL, the need for a NULL padded leading slash extension and the
> need to put control characters in the remaining bytes, we've only got 30
> bits to play with, we're never going to avoid collisions in a space of
> up to 31 bits.  Technically, a random function is at least as good at
> collision avoidance as any deterministic solution ... and it's a lot
> easier to code.

You could be deterministic and restrict maximum number of entries in
directory?

You could do random function but check if duplicate exists, and return
-EPERM if it does?

                                                                Pavel

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Pavel,

 > > I disagree with this: given there's been testing with no known issues, it's
 > > not a back compat question.
 >
 > The testing uncovered some pretty serious issues. ('Causes XP to
 > bluescreen').

no, that's not quite correct. I have never been able to produce a
bluescreen with the patch I have posted.

I did manage to produce a bluescreen with a different patch that
deliberately lowers the number of bits of randomness used (ie. just
change the mask to random32() in vfat_build_dummy_83_buffer()). Even
then, producing a bluescreen is not trivial, though it is definately
possible.

As I explained in a previous posting, the WindowsXP bug doesn't happen
just because two 8.3 entries have the same 11 bytes. There are
additional strong constraints on the access patterns by the WindowsXP
kernel before the bluescreen happens. I reproduced it by running 4
processes in parallel doing massive numbers of random file operations,
with a controlling script transferring control of the USB key between
the WindowsXP box and a Linux host every minute (so both are doing
random file operations).

Even with that, if I had even a few bits of randomness then it took a
lot of testing (often hours) to produce the bluescreen. I wouldn't
have even known the problem was possible if I hadn't been
experimenting with patch varients where all the bytes are always the
same (such as the 11 spaces varient I mentioned). With those varients
(where there is no randomness at all), you can produce a bluescreen
with a bit of effort on the DOS command line.

So I hope you can see why I don't think this is a particularly large
problem, especially given that the WindowsXP FAT driver is notoriously
prone to crashing, as a quick google search will show you.

If someone can actually demonstrate the bluescreen with the patch I've
posted in some access pattern that is at all likely to be made by a
WindowsXP user then I would be more concerned. As it is, I don't think
we should hold off on this patch for what amounts to a theoretical
issue.

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Pavel,

 > You already know that it breaks XP and older linuxes. So... what are
 > you arguing about?! Chkdsk marks it as invalid filesystem... not a
 > vfat.

no, wrong on all 3 counts.

For the "breaks XP", see my previous message where I point out that I
have never been able to crash XP with this patch, only with other
varients on the patch that reduce the number of random bits we use.

The "breaks older linuxes" claim is also wrong. The patch I have
posted works fine with all older versions of the Linux kernel that I
have tested. If you know of a old version of the Linux kernel that
doesn't work with a filesystem written when this patch is enabled then
please let me know. I think you may be getting confused by the
discussion I had with Eric where I explained about the reasons for the
change in fat/dir.c. I won't repeat that long discussion again here,
but please re-read the reply I sent to Eric and if you still have
questions about it then please ask.

Finally, saying that "chkdsk marks it as an invalid filesystem" is not
correct. It only complains if there happens (with a very low
probability!) to be two files with the same 11 bytes in their
respective 8.3 entries. When it complains it renames one of the two
files.

That rename would not in itself be a problem at all, as changing the
8.3 entry would not affect the long filename, except that chkdsk has a
bug where it doesn't follow the Microsoft FAT specification. When you
rename a file on a FAT filesystem you are supposed to also update the
8 bit checksum field in the corresponding long filename entries. If
you don't do that update then you effectively strip off the long
filename. Current versions of Microsoft chkdsk neglect to update the
long filename checksum when renaming after detecting a duplicate 8.3
entry.

So it is Microsoft's chkdsk, not the patched Linux kernel, that is in
violation of the FAT spec. Luckily the probability of this bug
cropping up is quite small in practice.

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Pavel Machek]

On Fri 2009-07-03 08:06:54, tridge@... wrote:

> Hi Pavel,
>
>  > > I disagree with this: given there's been testing with no known issues, it's
>  > > not a back compat question.
>  >
>  > The testing uncovered some pretty serious issues. ('Causes XP to
>  > bluescreen').
>
> no, that's not quite correct. I have never been able to produce a
> bluescreen with the patch I have posted.

So you know it causes XP to bluescreen, but can not reproduce that. So
what? Someone, somewhere _will_ reproduce it.
                                                                                Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Pavel,

 > So you know it causes XP to bluescreen, but can not reproduce that. So
 > what? Someone, somewhere _will_ reproduce it.

yes, like someone, somewhere will get data corruption in a TCP
connection because the checksum is quite weak and networking hardware
ain't perfect.

Once the probabilties become small enough then it is normal to accept
imperfection in operating systems. If I could make it perfect I would,
but I haven't thought of a way to do that while being absolutely sure
of maintaining the full strength of the legal defence.

If you find a way to actually produce this problem in a way that is
realistic for Linux users then let me know. Otherwise I think it is
the best option we have, imperfect though it is.

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Pavel Machek]

On Fri 2009-07-03 08:23:34, tridge@... wrote:

> Hi Pavel,
>
>  > You already know that it breaks XP and older linuxes. So... what are
>  > you arguing about?! Chkdsk marks it as invalid filesystem... not a
>  > vfat.
>
> no, wrong on all 3 counts.
>
> For the "breaks XP", see my previous message where I point out that I
> have never been able to crash XP with this patch, only with other
> varients on the patch that reduce the number of random bits we use.

...where you point out that probability of breakage is not too
high. But ammount of XP machines out there _is_ quite high, so someone
will hit it. You are deliberately breaking XP.

> The "breaks older linuxes" claim is also wrong. The patch I have
> posted works fine with all older versions of the Linux kernel that I
> have tested. If you know of a old version of the Linux kernel that
> doesn't work with a filesystem written when this patch is enabled then
> please let me know. I think you may be getting confused by the
> discussion I had with Eric where I explained about the reasons for the
> change in fat/dir.c. I won't repeat that long discussion again here,
> but please re-read the reply I sent to Eric and if you still have
> questions about it then please ask.

Ok, I guess I misunderstood. So what is the reason for pushing stable patch?

> Finally, saying that "chkdsk marks it as an invalid filesystem" is not
> correct. It only complains if there happens (with a very low
> probability!) to be two files with the same 11 bytes in their
> respective 8.3 entries. When it complains it renames one of the two
> files.

The "very low probability" is 50% for 30000 entries in directory,
AFAICT.

> That rename would not in itself be a problem at all, as changing the
> 8.3 entry would not affect the long filename, except that chkdsk has a
> bug where it doesn't follow the Microsoft FAT specification. When

Yeah, when map and reality disagree, trust the map. Not.

I don't care what M$ spec says, and you should not care, either. It is
M$ chkdsk that matters, and M$ VFAT implementation that matters.

> So it is Microsoft's chkdsk, not the patched Linux kernel, that is in
> violation of the FAT spec. Luckily the probability of this bug
> cropping up is quite small in practice.

Yes, explain that to the people. Actually, write it to the Kconfig.
"Turn this option ON to show bugs in M$ chkdsk".

                                                                        Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Pavel Machek]

On Fri 2009-07-03 08:41:13, tridge@... wrote:
> Hi Pavel,
>
>  > So you know it causes XP to bluescreen, but can not reproduce that. So
>  > what? Someone, somewhere _will_ reproduce it.
>
> yes, like someone, somewhere will get data corruption in a TCP
> connection because the checksum is quite weak and networking hardware
> ain't perfect.

That's why we have ethernet checksums.

> Once the probabilties become small enough then it is normal to accept
> imperfection in operating systems. If I could make it perfect I
> but I haven't thought of a way to do that while being absolutely sure
> of maintaining the full strength of the legal defence.

It already _was_ perfect before you started patching it.

> If you find a way to actually produce this problem in a way that is
> realistic for Linux users then let me know. Otherwise I think it is
> the best option we have, imperfect though it is.

So, you know that particular option will cause someone's XP to
bluescreen, and  you still want it to default to "Y"?
                                                                                        Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Jan Engelhardt-3]

On Wednesday 2009-07-01 16:05, Theodore Tso wrote:

>On Wed, Jul 01, 2009 at 12:25:58PM +0100, Alan Cox wrote:
>
>> (most *FAT using products don't use Microsofts
>> implementation). Testing it versus Windows and saying it works is
>> not really adequate. Thats what ACPI and BIOS people do that *we*
>> moan about all the time.
>
>[...]
>The other big user I can think of are digital cameras, but (a)
>normally most users read from them and then delete the pictures, and
>rarely write to media meant for a digital camera, and (b) the DCIM
>standard for digital cameras explicitly only supports 8.3 filenames
>and so digital camera manufacturers explicitly don't need to deal with
>Long File Names at all.  (Hmm.... I wonder why....)
>[...]
>Ultimately, though, requiring that every single possible device be
>tested is probably not reasonable, so the best way to do this testing
>is the way do most of our testing; we do basic due diligence, but then
>we merge it into mainline and let our huge user community try it out.

Yes, here is your precedent case. The DUALNAMES patch breaks the
operation on my Fujifilm Finepix A210 digital camera.

Here, dscf4159.jpg is a preexisting file created by the camera itself
(and subsequently, Finepix's FAT code) as a result of taking a picture.
Then I just copied that file with two different kernels in succession.

 * 2.6.31-rc1 + dualnames=n
   # mount /dev/sdc1 /mnt
   # cd /mnt/dcim/100_fuji/; cp dscf4159.jpg dscf3000.jpg
   # umount /mnt

 * 2.6.29.5 w/o patch
   # same procedure
   cp dscf4159.jpg dscf3001.jpg

End result is that picture ID 3000 is not selectable in the camera's
built-in menu (OSD), as if the file did not exist. 3001 was shown,
however.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Jan,

 > Yes, here is your precedent case. The DUALNAMES patch breaks the
 > operation on my Fujifilm Finepix A210 digital camera.

interesting, but I think you may have been trapped by differences in
the VFAT mounting options between the two examples.

Would you mind retesting again, but looking carefully at the VFAT
mount options. I don't think what you've described can happen without
differing VFAT options between your two examples.

I'll try and expain:

 > Here, dscf4159.jpg is a preexisting file created by the camera itself
 > (and subsequently, Finepix's FAT code) as a result of taking a picture.
 > Then I just copied that file with two different kernels in succession.

ok, dscf4159.jpg may or may not be a 8.3 name depending on the
"shortname=" mount option. As this is a digital camera, I suspect that
it was created as an 8.3 name, although you'd have to confirm that by
inspecting the disk image. If you made a copy of the image available
to me somewhere (please don't email it unless it compresses to a small
size!) then I can tell you what the camera did in this case.

 >  * 2.6.31-rc1 + dualnames=n
 >    # mount /dev/sdc1 /mnt
 >    # cd /mnt/dcim/100_fuji/; cp dscf4159.jpg dscf3000.jpg
 >    # umount /mnt

Now we would need to know what shortname= option was used here. If the
shortname= option was such that the lowercase name dscf3000.jpg was
considered as 8.3, then the patch I have posted would have had zero
effect on what was stored on disk. If it was not considered a short
name (because you have the shortname= option set to not allow
lowercase short names) then it would have been created as a long
name. As you said the file became invisible then I suspect that for
this test you had the shortname= option set to treat lowercase names
as not being allowed in 8.3.

 >  * 2.6.29.5 w/o patch
 >    # same procedure
 >    cp dscf4159.jpg dscf3001.jpg
 >
 > End result is that picture ID 3000 is not selectable in the camera's
 > built-in menu (OSD), as if the file did not exist. 3001 was shown,
 > however.

ok, if 3001 was shown, then in this case you must have had different
VFAT mount options. I don't know if this is because the different base
kernel has different defaults, or if perhaps you used a different
system that had different mounting options.

If you had used the same mount options as the previous test, then the
3001 name could not have shown up I think. It would have shown up as
something like DSCF3~01.JPG.

So I think all you've found is two things we already knew:

 1) VFAT mount options make for an easy trap to fall into :-)

 2) as the config text in the patch explains:

        "That means that long filenames created with this option
         disabled will not be accessible at all to operating systems
         that do not understand the VFAT extensions."

If you mount with the right shortname= options then I think you'll
find that your above test will work (for some value of work!)

But please do keep testing. We want to eliminate the possibility that
you have found a real example of where the patch causes a problem with
some embedded systems.

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Jan Engelhardt-3]

On Friday 2009-07-03 01:17, Jan Engelhardt wrote:

>
>On Wednesday 2009-07-01 16:05, Theodore Tso wrote:
>>On Wed, Jul 01, 2009 at 12:25:58PM +0100, Alan Cox wrote:
>>
>>> (most *FAT using products don't use Microsofts
>>> implementation). Testing it versus Windows and saying it works is
>>> not really adequate. Thats what ACPI and BIOS people do that *we*
>>> moan about all the time.
>>
>>[...]
>>The other big user I can think of are digital cameras, but (a)
>>normally most users read from them and then delete the pictures, and
>>rarely write to media meant for a digital camera, and (b) the DCIM
>>standard for digital cameras explicitly only supports 8.3 filenames
>>and so digital camera manufacturers explicitly don't need to deal with
>>Long File Names at all.  (Hmm.... I wonder why....)
>>[...]
>>Ultimately, though, requiring that every single possible device be
>>tested is probably not reasonable, so the best way to do this testing
>>is the way do most of our testing; we do basic due diligence, but then
>>we merge it into mainline and let our huge user community try it out.
>
>
>Yes, here is your precedent case. The DUALNAMES patch breaks the
>operation on my Fujifilm Finepix A210 digital camera.

Adding to that, I've just run it against every possible USB device I
could dig up in my room (i.e. one more for now).

It also breaks with the Tekmax T-1000/TMP-1000 (nicknamed
"ioneit"/"i-one-it"/"i-want-it") digital audio player. Its built-in
directory browser does display the copied file (see below), but does not
want to play it nor enqueue it into the playlist.

Again, a simple copy operation was performed — this time for both
8.3 and long names, i.e.:

  $ cp working.mp3 TEST0000.mp3
  $ cp working.mp3 TEST12345678.mp3

None of the dualnames=n-created files were accepted.

(Combinations that worked: 2.6.29.5-w/o-dualnames-patch, and
2.6.31-rc1+dualnames=y)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Rusty Russell]

On Fri, 3 Jul 2009 07:16:46 am Pavel Machek wrote:

> On Wed 2009-07-01 20:19:51, Rusty Russell wrote:
> > On Tue, 30 Jun 2009 04:01:03 pm Pavel Machek wrote:
> > > > +config VFAT_FS_DUALNAMES
> > > > + bool "VFAT dual names support"
> > > > + depends on VFAT_FS
> > >
> > > Defaults should be back-compatible.
> >
> > Hi Pavel,
> >
> > I disagree with this: given there's been testing with no known issues,
> > it's not a back compat question.
>
> The testing uncovered some pretty serious issues. ('Causes XP to
> bluescreen').

No, read more carefully: testing revealed "couldn't cause XP to bluescreen".  
It took a deliberately hobbled version of the patch to tickle the XP bug,
under a day of stress testing.

Cheers,
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Pavel,

 > That's why we have ethernet checksums.

which of course just changes the number of bits. The argument remains
the same.

 > It already _was_ perfect before you started patching it.

wow, I really didn't expect the objection to my patch to be that VFAT
is perfect!

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Rusty Russell]

On Fri, 3 Jul 2009 08:14:46 am Pavel Machek wrote:
> On Fri 2009-07-03 08:41:13, tridge@... wrote:
> > If you find a way to actually produce this problem in a way that is
> > realistic for Linux users then let me know. Otherwise I think it is
> > the best option we have, imperfect though it is.
>
> So, you know that particular option will cause someone's XP to
> bluescreen, and  you still want it to default to "Y"?

No, his point is the opposite: the rate of incidence of bluescreens on XP will
not measurably increase due to this patch.

Cheers,
Rusty.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Jan Engelhardt-3]

On Friday 2009-07-03 01:37, tridge@... wrote:
>
> > Yes, here is your precedent case. The DUALNAMES patch breaks the
> > operation on my Fujifilm Finepix A210 digital camera.
>
>interesting, but I think you may have been trapped by differences in
>the VFAT mounting options between the two examples.

No special options — all defaults to whatever the Linux kernel and
util-linux 2.14.1 gave me. Whereby util-linux should not make a
difference because it just passes on the mount request for there is
no special vfat helper.

Given my second report about the audio player (just sent out), I even
compared dualnames=n with =y on the same kernel — which would rule
out changed defaults between 2.6.29 and 2.6.31-rc.

/proc/mounts contains, on .31-rc:

/dev/sdc1 /mnt vfat rw,relatime,fmask=0022,dmask=0022,codepage=437
iocharset=utf8,errors=remont-ro 0 0

> > Here, dscf4159.jpg is a preexisting file created by the camera itself
> > (and subsequently, Finepix's FAT code) as a result of taking a picture.
> > Then I just copied that file with two different kernels in succession.
>
>ok, dscf4159.jpg may or may not be a 8.3 name depending on the
>"shortname=" mount option. As this is a digital camera, I suspect that
>it was created as an 8.3 name, although you'd have to confirm that by
>inspecting the disk image. If you made a copy of the image available
>to me somewhere (please don't email it unless it compresses to a small
>size!) then I can tell you what the camera did in this case.

http://jengelh.medozas.de/files/finepix.img.bz2
(Includes an unreadable dscf3000 entry.)

Then, I just tried everything to get an overview:

 (All with DUALNAMES=n)
 # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=lower
 # cp /mnt/dcim/100_fuji/dscf{4160,3004}.jpg
 # umount /mnt
 # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=win95
 # cp /mnt/dcim/100_fuji/dscf{4160,3005}.jpg
 # umount /mnt
 # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=winnt
 # cp /mnt/dcim/100_fuji/dscf{4160,3006}.jpg
 # umount /mnt
 # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=mixed
 # cp /mnt/dcim/100_fuji/dscf{4160,3007}.jpg
 # umount /mnt

Result? The camera only displays 3006.

The dualnames patch's filling filenames with random illegal
chars does seems to have a really foul side-effect after all.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Jan,

I should also mention that in the first patch I sent back in May, I
added some code that made the 8.3 name problem easier on the user. It
basically forced the shortname= option, overriding the mount option,
when the patch was triggered. The end result was that a name like
dcis3000.jpg would always be stored as a 8.3 name, and thus be visible
to digital cameras, even if the mount options were set to say that
lowercase names should be considered as "long" names.

Hirofumi objected, on the basis that I was overriding the VFAT
mount option. I think that overriding in this case would in fact be
worthwhile, as I don't think many users ever think carefully about
what shortname= option they are using, and unless you get it right
then you can get some real surprises.

Hirofumi-san, do you think that same objection you expressed
previously also applies to this new patch? For example, would you
think it reasonable to modify the patch to store the filename as 8.3
if it can fit in 8.3 in a case mapped manner? Or store as 8.3 if it is
all uppercase or all lowercase?

I think we will hit cases like the one Jan has pointed out where the
exising shortname= options will become even more confusing than they
were already when this new patch is applied, especially as I think
most distros these days magically supply a shortname= option when you
insert some VFAT formatted media.

Cheers, Tridge
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Hi Jan,

 > Then, I just tried everything to get an overview:

thanks!

 >  (All with DUALNAMES=n)
 >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=lower
 >  # cp /mnt/dcim/100_fuji/dscf{4160,3004}.jpg
 >  # umount /mnt
 >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=win95
 >  # cp /mnt/dcim/100_fuji/dscf{4160,3005}.jpg
 >  # umount /mnt
 >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=winnt
 >  # cp /mnt/dcim/100_fuji/dscf{4160,3006}.jpg
 >  # umount /mnt
 >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=mixed
 >  # cp /mnt/dcim/100_fuji/dscf{4160,3007}.jpg
 >  # umount /mnt
 >
 > Result? The camera only displays 3006.
 >
 > The dualnames patch's filling filenames with random illegal
 > chars does seems to have a really foul side-effect after

actually, you've just proved the patch works as intended on your
device!

Only the shortname=winnt would create a 8.3 name when you use the
filename dscfNNNN.jpg. That is why my first patch in May forced

   shortname_flags = VFAT_SFN_CREATE_WINNT;

when the patch triggered. I think this is the only sane behaviour for
a VFAT filesystem on Linux. Unfortunately it isn't the default, and at
least on my Ubuntu system the automatic mounting doesn't get this
right.

I'd love to change the default or do an override, at least when
dualnames is diabled. Hirofumi-san, what do you think?

Cheers, Tridge

PS: I'm assuming you actually typed "shortname=" not
"shortnames=". The option is not plural.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[OGAWA Hirofumi]

tridge@... writes:

> I should also mention that in the first patch I sent back in May, I
> added some code that made the 8.3 name problem easier on the user. It
> basically forced the shortname= option, overriding the mount option,
> when the patch was triggered. The end result was that a name like
> dcis3000.jpg would always be stored as a 8.3 name, and thus be visible
> to digital cameras, even if the mount options were set to say that
> lowercase names should be considered as "long" names.
>
> Hirofumi objected, on the basis that I was overriding the VFAT
> mount option. I think that overriding in this case would in fact be
> worthwhile, as I don't think many users ever think carefully about
> what shortname= option they are using, and unless you get it right
> then you can get some real surprises.
>
> Hirofumi-san, do you think that same objection you expressed
> previously also applies to this new patch? For example, would you
> think it reasonable to modify the patch to store the filename as 8.3
> if it can fit in 8.3 in a case mapped manner? Or store as 8.3 if it is
> all uppercase or all lowercase?

I have no objection to improve. And in this case, for this option, to
force the shortname= sounds reasonable.

Thanks.

> I think we will hit cases like the one Jan has pointed out where the
> exising shortname= options will become even more confusing than they
> were already when this new patch is applied, especially as I think
> most distros these days magically supply a shortname= option when you
> insert some VFAT formatted media.
--
OGAWA Hirofumi <hirofumi@...>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Jan Engelhardt-3]

On Friday 2009-07-03 02:25, tridge@... wrote:

>
> >  (All with DUALNAMES=n)
> >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=lower
> >  # cp /mnt/dcim/100_fuji/dscf{4160,3004}.jpg
> >  # umount /mnt
> >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=win95
> >  # cp /mnt/dcim/100_fuji/dscf{4160,3005}.jpg
> >  # umount /mnt
> >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=winnt
> >  # cp /mnt/dcim/100_fuji/dscf{4160,3006}.jpg
> >  # umount /mnt
> >  # mount /dev/sdc1 /mnt -o iocharset=utf8,shortnames=mixed
> >  # cp /mnt/dcim/100_fuji/dscf{4160,3007}.jpg
> >  # umount /mnt
> >
> > Result? The camera only displays 3006.
> >
> > The dualnames patch's filling filenames with random illegal
> > chars does seems to have a really foul side-effect after
>
>actually, you've just proved the patch works as intended on your
>device!
>
>Only the shortname=winnt would create a 8.3 name when you use the
>filename dscfNNNN.jpg.

The mount(8) manpage becomes pretty ambiguous with the dualnames
patch. By default, "store a long name when the short name is not all
upper case" would apply - and is invoked because I am a lazy typist
who wrote "cp $this dscf3000" instead of "cp $this DSCF3000". Now
since dualnames=n is in effect, no long name would need to be written
because it still fits into 8.3 — subsequently, one would assume an
8.3 is generated for all shortname= cases.

>That is why my first patch in May forced
>
>   shortname_flags = VFAT_SFN_CREATE_WINNT;

Making WINNT the default would cause many a `ls` output to just
scream at me in uppercaps because there are programs that
create long names with all-uppers.

shortname really needs to be split into "shortname that is applied
during creation" and "shortname that is applied upon readdir".

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/