gjournal questions and observations

[Ross Gohlke]

KUDOS
Congratulations to all GEOM contributors. While I am new to GEOM, so far
I am very impressed with the way it is designed and the capabilities
(both realized and anticipated) the design offers.

QUESTIONS
1. What is the best way to journal whole disks whose slices (without
partitions) are used by gconcat and gmirror? Does the same apply for gvinum?
The ultimate scenario seems to be journaling another GEOM class such as
gmirror because gjournal handles the synchronization of all mirror
consumers. You can turn off autosync on the mirror, thus saving CPU
cycles and improving disk access. (Am I right?)

2. How should gjournal and gmirror be configured when the journal is
outside, instead of inside, the mirror?
The above scenario only seems possible if a) you are willing to journal
slices, which is not best practice [1] [2] or b) you use whole disks in
your mirrors, which is not very realistic.
Further I am on PowerPC and don't even have bsdlabel, so journaling
slices and mirroring partitions is not an option anyway.
My thought was to journal each disk separately, outside the mirror, and
keep autosynchronization on for the mirror.

[1]
http://lists.freebsd.org/pipermail/freebsd-questions/2008-April/173501.html
[2]
http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-11/msg00247.html

3. What is the best way to completely remove a whole disk journal such
that re-issuing
% gjournal label /dev/ad0
does not require -f?
I have tried gpart destroy/create and newfs -E. I have not tried
blanking the whole disk with dd, nor have I tried newfs -E on the whole
disk.

4. Does it matter whether gjournal is loaded when gjournal label is issued?
Originally I was journaling slices, and I was unable to properly stop a
particular slice.
% gjournal stop ad0s6.journal
% gjournal list
Showed the slice still loaded, but under a different name:
ie, ufsid/48x6x1bxc39394x7
I don't know if it was related to the originally issued gjournal label
command or that gjournal was loaded before the slice was labeled.
I believe what fixed it was:
% dd if=/dev/zero of=/dev/ad0s6

OBSERVATIONS
  GJOURNAL
These may seem obvious to you but were hard won by me.
1. Never try to store a whole disk's journal on a slice of the same disk.
2. When journaling a whole disk and keeping the journal on the same
disk, not only must the very last sector of the disk be reserved, but
also the last gig (or whatever the size of your journal). Just leave it
as free space when you create slices with gpart.
While gjournal man page states journaling an existing file system
REQUIRES a separate device for storing the journal, it appears to work
without specifying a second device. At least
% gjournal label -f /dev/ad0
seems to work, using the end of /dev/ad0 to store the journal whether a
  slice occupies those sectors or not. (Consequently, trying to gmirror
the last slice when it occupies journal sectors will fail.)

Regards,
Ross Gohlke
_______________________________________________
freebsd-geom@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "freebsd-geom-unsubscribe@..."

[Ross Gohlke]

> QUESTIONS
> 3. What is the best way to completely remove a whole disk journal such
> that re-issuing
> % gjournal label /dev/ad0
> does not require -f?

Blanking the beginning of the drive did the trick.
% dd if=/dev/zero of=/dev/ad0 bs=1m count=600

> 1. What is the best way to journal whole disks whose slices (without
> partitions) are used by gconcat and gmirror? Does the same apply for
> gvinum?

% gjournal label /dev/ad0
% gjournal load
   (Don't newfs the journal; wait for concat)
% gconcat load
% gconcat label ports ad0.journals11
% newfs /dev/concat/ports
% mount /dev/concat/ports /mnt/ports
Add new slices to the journal, not the disk.
% gpart add -i 12 ad0.journal

Setting up a gmirrored gconcat did not work. Another guy tried
gconcatted gmirrors and had a similar result [1]. He was able to fix the
problem with fsck, which did not work for me:
[1]
http://docs.freebsd.org/cgi/getmsg.cgi?fetch=0+0+archive/2006/freebsd-geom/20060806.freebsd-geom

This was the result when trying to rsync /usr/ports to /mnt/ports
(/dev/mirror/ports).
#/var/log/messages
...
Nov  9 17:00:51 rorty kernel: GEOM_CONCAT: Device ports created
(id=1362796578).
Nov  9 17:00:51 rorty kernel: GEOM_CONCAT: Disk ad0s11 attached to ports.
Nov  9 17:00:51 rorty kernel: GEOM_CONCAT: Device ports activated.
Nov  9 17:00:51 rorty kernel: GEOM_MIRROR: Device ports: provider
concat/ports marked as inactive, skipping.
Nov  9 17:00:51 rorty kernel: GEOM_MIRROR: Device ports: provider
ufsid/4af8442b031fbe48 marked as inactive, skipping.
Nov  9 17:00:59 rorty kernel: GEOM_MIRROR: Device mirror/ports launched
(1/1).
Nov  9 17:01:57 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry
left) LBA=116413904
Nov  9 17:02:50 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry
left) LBA=115497232
Nov  9 17:02:50 rorty kernel: ad0: TIMEOUT - WRITE_MUL retrying (1 retry
left) LBA=19936658
Nov  9 17:02:55 rorty kernel: ad1: TIMEOUT - READ_MUL retrying (1 retry
left) LBA=115498352
Nov  9 17:03:10 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073758208, length=2048)]error = 5
Nov  9 17:03:10 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5
Nov  9 17:03:10 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073762304, length=2048)]error = 5
Nov  9 17:03:41 rorty kernel: initiate_write_filepage: already started
Nov  9 17:03:41 rorty kernel:
initiga_tvef_sw_rdiotnee_(f)i:lmeiprargoer:/ paolrrtesa[dWyR
IsTtEa(rotfefdset
Nov  9 17:03:41 rorty kernel: =1i0n7i3t7i6a2t3e0_4w,r
ilteen_gftihl=e2p0a4g8e):] earlrroera d=y  5st
Nov  9 17:03:41 rorty kernel: artedg_v
Nov  9 17:03:41 rorty kernel:
fs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5
Nov  9 17:03:41 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073758208, length=2048)]error = 5
Nov  9 17:03:42 rorty kernel: initiate_write_filepage: already started
Nov  9 17:03:42 rorty kernel: initiate_write_filepage: already started
Nov  9 17:03:42 rorty kernel: initiate_wgr_ivtfes__fdiolneep(a)g:em:i
rarlorre/apdoyr tsst[aWrRtIeTdE(o
Nov  9 17:03:42 rorty kernel: ffset=1073758208, length=2048)]error = 5
Nov  9 17:03:42 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073760256, length=2048)]error = 5
Nov  9 17:03:42 rorty kernel:
g_vfs_done():mirror/ports[WRITE(offset=1073762304, length=2048)]error = 5
Nov  9 17:04:12 rorty kernel: initiate_write_filepage: already started
Nov  9 17:04:12 rorty kernel:
ingit_ivaftse__dworniet(e)_:fmiilrerpoarg/ep:o ratlsr[eWaRdIyT
Es(toafrftseedt=1
Nov  9 17:04:12 rorty kernel: 073i7n6i2t3i0a4t,e
_lwernigtteh_=fi2l0e4p8a)g]ee:r raolrr e=a d5y
Nov  9 17:04:12 rorty kernel: startge_dvf
Nov  9 17:04:12 rorty kernel:
s_done():mirror/ports[WRIiTnEi(toifaftsee_tw=ri1t0e7_3f7i6l0e2p5a6g,e :l
eanlgrteha=d2y0 4s8t)$
Nov  9 17:04:12 rorty kernel: = 5
Nov  9 17:04:12 rorty kernel: g_vfs_done():mirror/ports[WRIT
...

I have ruled out the disk being bad because it works fine with gmirror
OR gconcat, just not both.

Regards,
Ross
_______________________________________________
freebsd-geom@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-geom
To unsubscribe, send any mail to "freebsd-geom-unsubscribe@..."

[Pawel Jakub Dawidek]

On Sat, Nov 07, 2009 at 10:56:29PM -0600, Ross Gohlke wrote:

> KUDOS
> Congratulations to all GEOM contributors. While I am new to GEOM, so far
> I am very impressed with the way it is designed and the capabilities
> (both realized and anticipated) the design offers.
>
> QUESTIONS
> 1. What is the best way to journal whole disks whose slices (without
> partitions) are used by gconcat and gmirror? Does the same apply for gvinum?
> The ultimate scenario seems to be journaling another GEOM class such as
> gmirror because gjournal handles the synchronization of all mirror
> consumers. You can turn off autosync on the mirror, thus saving CPU
> cycles and improving disk access. (Am I right?)

You should always gjournal top-most provider, so you always put UFS on
top of <name>.journal provider. Don't do anything with <name>.journal
besides of file system configuration.

> 2. How should gjournal and gmirror be configured when the journal is
> outside, instead of inside, the mirror?
> The above scenario only seems possible if a) you are willing to journal
> slices, which is not best practice [1] [2] or b) you use whole disks in
> your mirrors, which is not very realistic.
> Further I am on PowerPC and don't even have bsdlabel, so journaling
> slices and mirroring partitions is not an option anyway.
> My thought was to journal each disk separately, outside the mirror, and
> keep autosynchronization on for the mirror.
>
> [1]
> http://lists.freebsd.org/pipermail/freebsd-questions/2008-April/173501.html
> [2]
> http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2008-11/msg00247.html

See above.

You can safely gmirror disks, slices or partition and put gjournal on
top of gmirror and file system on top of gjournal.

> 3. What is the best way to completely remove a whole disk journal such
> that re-issuing
> % gjournal label /dev/ad0
> does not require -f?
> I have tried gpart destroy/create and newfs -E. I have not tried
> blanking the whole disk with dd, nor have I tried newfs -E on the whole
> disk.

gjournal stop <name>.journal (or <name>); gjournal clear <name>

> 4. Does it matter whether gjournal is loaded when gjournal label is issued?
> Originally I was journaling slices, and I was unable to properly stop a
> particular slice.
> % gjournal stop ad0s6.journal
> % gjournal list
> Showed the slice still loaded, but under a different name:
> ie, ufsid/48x6x1bxc39394x7

You provider is accessible by few different names. This ufsid/ thing
(which I don't like) is one of them. Once you stop gjournal on one name
it is recreated using another name. Besides of using -h option to
gjournal label and hardcoding provider name there is not much we can do.

> While gjournal man page states journaling an existing file system
> REQUIRES a separate device for storing the journal, it appears to work
> without specifying a second device. At least
> % gjournal label -f /dev/ad0
> seems to work, using the end of /dev/ad0 to store the journal whether a
>  slice occupies those sectors or not. (Consequently, trying to gmirror
> the last slice when it occupies journal sectors will fail.)

It will eventually work until your UFS will start to use space gjournal
is using for journal. Absolutely don't do that. Its like creating 4GB
file system on 3GB provider - at some point you will need the missing
1GB..

--
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd@...                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!