Nabble - linux-ext4

Re: [PATCH] ext4: return correct wbc.nr_to_write in ext4_da_writepages

2009-12-17T09:32:32Z

On Thu, Dec 17, 2009 at 09:40:25AM -0600, Eric Sandeen wrote:

> Richard Kennedy wrote:
> > ext4: always re-base nr_to_write in ext4_da_writepages
> >
> > When ext4_da_writepages increases the nr_to_write in writeback_control
> > then it must always re-base the return value.
> >
> > Without this change, when wb_writeback calculates how many pages were
> > actually written it can get a negative value and loop more times than
> > necessary. In tests I have seen nearly all the dirty pages pushed out to
> > writeback due to this issue.
> >
> > Signed-off-by: Richard Kennedy <richard@...>
> >
> > ----
> >
> > patch against 2.6.32
> > tested on x86_64
> >
> > wb_writeback calculates (MAX_WRITE_PAGES - nr_to_write) & cannot know
> > that the value got changed.
> >
> > I'm not sure what the test I removed was for.
> > Perhaps
> > if (nr_to_writebump)
> > wbc->nr_to_write -= nr_to_writebump;
> > was intended?
>
> Ted's commit 55138e0b added it (just part of the commit):
>
> @@ -2914,7 +2994,8 @@ retry:
> out_writepages:
> if (!no_nrwrite_index_update)
> wbc->no_nrwrite_index_update = 0;
> - wbc->nr_to_write -= nr_to_writebump;
> + if (wbc->nr_to_write > nr_to_writebump)
> + wbc->nr_to_write -= nr_to_writebump;
> wbc->range_start = range_start;
> trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
> return ret;
>
> so it looks like the intent there was to stop ->nr_to_write from
> going negative ...

I guess writeback code can handle nr_to_write going negative. If we are
not updating wbc->nr_to_write then i guess writeback code will get a
wrong value for number of pages written and can end up doing wrong things
We had it that way as a part of 22208dedbd7626e5fc4339c417f8d24cc21f79d7
and i guess we didn't had any problems with that

So for the patch

Acked-by: Aneesh Kumar K.V <aneesh.kumar@...>

-aneesh
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] quota: manage reserved space when quota is not active [v2]

2009-12-17T08:14:37Z

On Thu 17-12-09 18:25:55, Dmitry Monakhov wrote:

> Since we implemented generic reserved space management interface,
> then it is possible to account reserved space even when quota
> is not active (similar to i_blocks/i_bytes).
>
> Without this patch following testcase result in massive comlain from
> WARN_ON in dquot_claim_space()
>
> TEST_CASE:
> mount /dev/sdb /mnt -oquota
> dd if=/dev/zero of=/mnt/test bs=1M count=1
> quotaon /mnt
> # fs_reserved_spave == 1Mb
> # quota_reserved_space == 0, because quota was disabled
> dd if=/dev/zero of=/mnt/test seek=1 bs=1M count=1
> # fs_reserved_spave == 2Mb
> # quota_reserved_space == 1Mb
> sync # ->dquot_claim_space() -> WARN_ON

Thanks. Folded into "quota: decouple fs reserved space from quota
reservation"

Honza

>
> Signed-off-by: Dmitry Monakhov <dmonakhov@...>
> ---
> fs/quota/dquot.c | 10 ++++++----
> include/linux/quotaops.h | 11 +++++++++--
> 2 files changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
> index dea86ab..a532d44 100644
> --- a/fs/quota/dquot.c
> +++ b/fs/quota/dquot.c
> @@ -1351,28 +1351,30 @@ static qsize_t *inode_reserved_space(struct inode * inode)
> return inode->i_sb->dq_op->get_reserved_space(inode);
> }
>
> -static void inode_add_rsv_space(struct inode *inode, qsize_t number)
> +void inode_add_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) += number;
> spin_unlock(&inode->i_lock);
> }
> +EXPORT_SYMBOL(inode_add_rsv_space);
>
> -
> -static void inode_claim_rsv_space(struct inode *inode, qsize_t number)
> +void inode_claim_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) -= number;
> __inode_add_bytes(inode, number);
> spin_unlock(&inode->i_lock);
> }
> +EXPORT_SYMBOL(inode_claim_rsv_space);
>
> -static void inode_sub_rsv_space(struct inode *inode, qsize_t number)
> +void inode_sub_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) -= number;
> spin_unlock(&inode->i_lock);
> }
> +EXPORT_SYMBOL(inode_sub_rsv_space);
>
> static qsize_t inode_get_rsv_space(struct inode *inode)
> {
> diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
> index 3ebb231..a529d86 100644
> --- a/include/linux/quotaops.h
> +++ b/include/linux/quotaops.h
> @@ -26,6 +26,10 @@ static inline void writeout_quota_sb(struct super_block *sb, int type)
> sb->s_qcop->quota_sync(sb, type);
> }
>
> +void inode_add_rsv_space(struct inode *inode, qsize_t number);
> +void inode_claim_rsv_space(struct inode *inode, qsize_t number);
> +void inode_sub_rsv_space(struct inode *inode, qsize_t number);
> +
> int dquot_initialize(struct inode *inode, int type);
> int dquot_drop(struct inode *inode);
> struct dquot *dqget(struct super_block *sb, unsigned int id, int type);
> @@ -42,7 +46,6 @@ int dquot_alloc_inode(const struct inode *inode, qsize_t number);
> int dquot_reserve_space(struct inode *inode, qsize_t number, int prealloc);
> int dquot_claim_space(struct inode *inode, qsize_t number);
> void dquot_release_reserved_space(struct inode *inode, qsize_t number);
> -qsize_t dquot_get_reserved_space(struct inode *inode);
>
> int dquot_free_space(struct inode *inode, qsize_t number);
> int dquot_free_inode(const struct inode *inode, qsize_t number);
> @@ -199,6 +202,8 @@ static inline int vfs_dq_reserve_space(struct inode *inode, qsize_t nr)
> if (inode->i_sb->dq_op->reserve_space(inode, nr, 0) == NO_QUOTA)
> return 1;
> }
> + else
> + inode_add_rsv_space(inode, nr);
> return 0;
> }
>
> @@ -221,7 +226,7 @@ static inline int vfs_dq_claim_space(struct inode *inode, qsize_t nr)
> if (inode->i_sb->dq_op->claim_space(inode, nr) == NO_QUOTA)
> return 1;
> } else
> - inode_add_bytes(inode, nr);
> + inode_claim_rsv_space(inode, nr);
>
> mark_inode_dirty(inode);
> return 0;
> @@ -235,6 +240,8 @@ void vfs_dq_release_reservation_space(struct inode *inode, qsize_t nr)
> {
> if (sb_any_quota_active(inode->i_sb))
> inode->i_sb->dq_op->release_rsv(inode, nr);
> + else
> + inode_sub_rsv_space(inode, nr);
> }
>
> static inline void vfs_dq_free_space_nodirty(struct inode *inode, qsize_t nr)
> --
> 1.6.0.4
>

--
Jan Kara <jack@...>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: return correct wbc.nr_to_write in ext4_da_writepages

2009-12-17T07:58:26Z

On Thu, 2009-12-17 at 09:40 -0600, Eric Sandeen wrote:

wb_writeback is OK with negative, it just needs to know how many pages
were written. Then it can decide if it's done the work it was asked to
do. balance_dirty_pages uses this throttle a device by asking for
writeback on a small number of pages.
regards
Richard

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: return correct wbc.nr_to_write in ext4_da_writepages

2009-12-17T07:40:25Z

Richard Kennedy wrote:

> ext4: always re-base nr_to_write in ext4_da_writepages
>
> When ext4_da_writepages increases the nr_to_write in writeback_control
> then it must always re-base the return value.
>
> Without this change, when wb_writeback calculates how many pages were
> actually written it can get a negative value and loop more times than
> necessary. In tests I have seen nearly all the dirty pages pushed out to
> writeback due to this issue.
>
> Signed-off-by: Richard Kennedy <richard@...>
>
> ----
>
> patch against 2.6.32
> tested on x86_64
>
> wb_writeback calculates (MAX_WRITE_PAGES - nr_to_write) & cannot know
> that the value got changed.
>
> I'm not sure what the test I removed was for.
> Perhaps
> if (nr_to_writebump)
> wbc->nr_to_write -= nr_to_writebump;
> was intended?

Ted's commit 55138e0b added it (just part of the commit):

@@ -2914,7 +2994,8 @@ retry:
out_writepages:
if (!no_nrwrite_index_update)
wbc->no_nrwrite_index_update = 0;
- wbc->nr_to_write -= nr_to_writebump;
+ if (wbc->nr_to_write > nr_to_writebump)
+ wbc->nr_to_write -= nr_to_writebump;
wbc->range_start = range_start;
trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
return ret;

so it looks like the intent there was to stop ->nr_to_write from
going negative ...

> regards
> Richard
>
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 2c8caa5..52a573c 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -2999,8 +2999,7 @@ retry:
> out_writepages:
> if (!no_nrwrite_index_update)
> wbc->no_nrwrite_index_update = 0;
> - if (wbc->nr_to_write > nr_to_writebump)
> - wbc->nr_to_write -= nr_to_writebump;
> + wbc->nr_to_write -= nr_to_writebump;
> wbc->range_start = range_start;
> trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
> return ret;
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] quota: manage reserved space when quota is not active [v2]

2009-12-17T07:25:55Z

Since we implemented generic reserved space management interface,
then it is possible to account reserved space even when quota
is not active (similar to i_blocks/i_bytes).

Without this patch following testcase result in massive comlain from
WARN_ON in dquot_claim_space()

TEST_CASE:
mount /dev/sdb /mnt -oquota
dd if=/dev/zero of=/mnt/test bs=1M count=1
quotaon /mnt
# fs_reserved_spave == 1Mb
# quota_reserved_space == 0, because quota was disabled
dd if=/dev/zero of=/mnt/test seek=1 bs=1M count=1
# fs_reserved_spave == 2Mb
# quota_reserved_space == 1Mb
sync # ->dquot_claim_space() -> WARN_ON

Signed-off-by: Dmitry Monakhov <dmonakhov@...>
---
fs/quota/dquot.c | 10 ++++++----
include/linux/quotaops.h | 11 +++++++++--
2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
index dea86ab..a532d44 100644
--- a/fs/quota/dquot.c
+++ b/fs/quota/dquot.c
@@ -1351,28 +1351,30 @@ static qsize_t *inode_reserved_space(struct inode * inode)
return inode->i_sb->dq_op->get_reserved_space(inode);
}

-static void inode_add_rsv_space(struct inode *inode, qsize_t number)
+void inode_add_rsv_space(struct inode *inode, qsize_t number)
{
spin_lock(&inode->i_lock);
*inode_reserved_space(inode) += number;
spin_unlock(&inode->i_lock);
}
+EXPORT_SYMBOL(inode_add_rsv_space);

-
-static void inode_claim_rsv_space(struct inode *inode, qsize_t number)
+void inode_claim_rsv_space(struct inode *inode, qsize_t number)
{
spin_lock(&inode->i_lock);
*inode_reserved_space(inode) -= number;
__inode_add_bytes(inode, number);
spin_unlock(&inode->i_lock);
}
+EXPORT_SYMBOL(inode_claim_rsv_space);

-static void inode_sub_rsv_space(struct inode *inode, qsize_t number)
+void inode_sub_rsv_space(struct inode *inode, qsize_t number)
{
spin_lock(&inode->i_lock);
*inode_reserved_space(inode) -= number;
spin_unlock(&inode->i_lock);
}
+EXPORT_SYMBOL(inode_sub_rsv_space);

static qsize_t inode_get_rsv_space(struct inode *inode)
{
diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
index 3ebb231..a529d86 100644
--- a/include/linux/quotaops.h
+++ b/include/linux/quotaops.h
@@ -26,6 +26,10 @@ static inline void writeout_quota_sb(struct super_block *sb, int type)
sb->s_qcop->quota_sync(sb, type);
}

+void inode_add_rsv_space(struct inode *inode, qsize_t number);
+void inode_claim_rsv_space(struct inode *inode, qsize_t number);
+void inode_sub_rsv_space(struct inode *inode, qsize_t number);
+
int dquot_initialize(struct inode *inode, int type);
int dquot_drop(struct inode *inode);
struct dquot *dqget(struct super_block *sb, unsigned int id, int type);
@@ -42,7 +46,6 @@ int dquot_alloc_inode(const struct inode *inode, qsize_t number);
int dquot_reserve_space(struct inode *inode, qsize_t number, int prealloc);
int dquot_claim_space(struct inode *inode, qsize_t number);
void dquot_release_reserved_space(struct inode *inode, qsize_t number);
-qsize_t dquot_get_reserved_space(struct inode *inode);

int dquot_free_space(struct inode *inode, qsize_t number);
int dquot_free_inode(const struct inode *inode, qsize_t number);
@@ -199,6 +202,8 @@ static inline int vfs_dq_reserve_space(struct inode *inode, qsize_t nr)
if (inode->i_sb->dq_op->reserve_space(inode, nr, 0) == NO_QUOTA)
return 1;
}
+ else
+ inode_add_rsv_space(inode, nr);
return 0;
}

@@ -221,7 +226,7 @@ static inline int vfs_dq_claim_space(struct inode *inode, qsize_t nr)
if (inode->i_sb->dq_op->claim_space(inode, nr) == NO_QUOTA)
return 1;
} else
- inode_add_bytes(inode, nr);
+ inode_claim_rsv_space(inode, nr);

mark_inode_dirty(inode);
return 0;
@@ -235,6 +240,8 @@ void vfs_dq_release_reservation_space(struct inode *inode, qsize_t nr)
{
if (sb_any_quota_active(inode->i_sb))
inode->i_sb->dq_op->release_rsv(inode, nr);
+ else
+ inode_sub_rsv_space(inode, nr);
}

static inline void vfs_dq_free_space_nodirty(struct inode *inode, qsize_t nr)
--
1.6.0.4

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] ext4: return correct wbc.nr_to_write in ext4_da_writepages

2009-12-17T07:17:43Z

ext4: always re-base nr_to_write in ext4_da_writepages

When ext4_da_writepages increases the nr_to_write in writeback_control
then it must always re-base the return value.

Without this change, when wb_writeback calculates how many pages were
actually written it can get a negative value and loop more times than
necessary. In tests I have seen nearly all the dirty pages pushed out to
writeback due to this issue.

Signed-off-by: Richard Kennedy <richard@...>

----

patch against 2.6.32
tested on x86_64

wb_writeback calculates (MAX_WRITE_PAGES - nr_to_write) & cannot know
that the value got changed.

I'm not sure what the test I removed was for.
Perhaps
if (nr_to_writebump)
wbc->nr_to_write -= nr_to_writebump;
was intended?

regards
Richard

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 2c8caa5..52a573c 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -2999,8 +2999,7 @@ retry:
out_writepages:
if (!no_nrwrite_index_update)
wbc->no_nrwrite_index_update = 0;
- if (wbc->nr_to_write > nr_to_writebump)
- wbc->nr_to_write -= nr_to_writebump;
+ wbc->nr_to_write -= nr_to_writebump;
wbc->range_start = range_start;
trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
return ret;

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] quota: manage reserved space even when quota is not active

2009-12-17T06:28:08Z

On Thu 17-12-09 03:35:25, Dmitry Monakhov wrote:

> Hi, Jan.
> I've prepared additional patch for quota reserved space management.
> Initially I've overlooked case when quota is not active, but in fact
> it seems reasonable to generalize reservation to that case as we do
> for i_block/i_bytes reservation.
> Possibly it should be folded in to c35fc5af93955aa22efc6568e5ae5b2cadbc086f
>
> CHANGE_LOG:
> quota: manage reserved space even when quota is not active
> Since we implemented generic reserved space management interface,
> then it is possible to account reserved space even quota when quota
> is not active (similar to i_blocks/i_bytes).

Yes, that's reasonable. I'll fold the patch into your
"quota: decouple fs reserved space from quota reservation" but please fix
things I write below.

> diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
> index dea86ab..de7e7de 100644
> --- a/fs/quota/dquot.c
> +++ b/fs/quota/dquot.c
> @@ -1351,15 +1351,14 @@ static qsize_t *inode_reserved_space(struct inode * inode)
> return inode->i_sb->dq_op->get_reserved_space(inode);
> }
>
> -static void inode_add_rsv_space(struct inode *inode, qsize_t number)
> +void inode_add_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) += number;
> spin_unlock(&inode->i_lock);
> }
>
> -
> -static void inode_claim_rsv_space(struct inode *inode, qsize_t number)
> +void inode_claim_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) -= number;
> @@ -1367,14 +1366,14 @@ static void inode_claim_rsv_space(struct inode *inode, qsize_t number)
> spin_unlock(&inode->i_lock);
> }
>
> -static void inode_sub_rsv_space(struct inode *inode, qsize_t number)
> +void inode_sub_rsv_space(struct inode *inode, qsize_t number)
> {
> spin_lock(&inode->i_lock);
> *inode_reserved_space(inode) -= number;
> spin_unlock(&inode->i_lock);
> }

The above three functions need to be exported (needed when ext4 is a
module).

> -static qsize_t inode_get_rsv_space(struct inode *inode)
> +qsize_t inode_get_rsv_space(struct inode *inode)
I think this does not need to be visible outside.

Honza
--
Jan Kara <jack@...>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: journalled quota seed-up

2009-12-17T02:07:24Z

Dmitry Monakhov <dmonakhov@...> writes:

> Dmitry Monakhov <dmonakhov@...> writes:
>
> Sorry forgot to add linux-ext4@... list to CC

>> Currently each quota modification result in write_dquot()
>> and later dquot_commit(). This means what each quota modification
>> function must wait for dqio_mutex. Which is *huge* performance
>> penalty on big SMP systems.
performance results:
simple tests-case: pwrite(,,40960,0); ftruncate(,0) in a loop
measured value: ops/ms == loops per msecond
mount: with journaled quota, and nodelalloc in order to force
curspace modification for each operation.
Seems it will be reasonable to prepare same patch for ext3.
| write-truncate | w/o quot | w jquot | w jquot mk_drt patch |
| nodelalloc | | | |
| NR tasks 1 | 20.4 | 14.5 | 14.83 |
| | 20.3 | 14.7 | 14.7 |
| | 20.4 | 14.8 | 14.8 |
| | | | |
| 4 | 6.1 | 4.76 | 6.3 |
| | 6.3 | 5.1 | 6.1 |
| | 6.5 | 4.8 | 6.3 |
| | | | |
| 8 | 3.75 | 2.59 | 3.34 |
| | 4.0 | 2.6 | 3.44 |
| | 3.76 | 2.63 | 3.47 |

>> ASAIU The core idea of this implementation
>> is to guarantee that each quota modification will be written to journal
>> atomically. But in fact this is not always true, because dquot may be
>> changed after dquot modification, but before it was committed to disk.
>>
>> | Task 1 | Task 2 |
>> | alloc_space(nr) | |
>> | ->spin_lock(dq_data_lock) | |
>> | ->curspace += nr | |
>> | ->spin_unlock(dq_data_lock) | |
>> | ->mark_dirty() | free_space(nr) |
>> | -->write_dquot() | ->spin_lock(dq_data_lock) |
>> | --->dquot_commit() | ->curspace -= nr |
>> | ---->commit_dqblk() | ->spin_unlock(dq_data_lock) |
>> | ----->spin_lock(dq_data_lock) | |
>> | ----->mem2disk_dqblk(ddq, dquot) | <<< Copy updated value |
>> | ----->spin_unlock(dq_data_lock) | |
>> | ----->quota_write() | |
>> Quota corruption not happens only because quota modification caller
>> started journal already. And ext3/4 allow only one running quota
>> at a time. Let's exploit this fact and avoid writing quota each time.
>> Act similar to dirty_for_io in general write_back path in page-cache.
>> If we have found what other task already started on copy and write the
>> dquot then we skip actual quota_write stage. And let that task do the job.
>> Side effect: Task which skip real quota_write() will not get an error
>> (if any). But this is not big deal because:
>> 1) Any error has global consequences (RO_remount, err_print, etc).
>> 2) Real IO is differed till the journall_commit.
>>
>> Signed-off-by: Dmitry Monakhov <dmonakhov@...>
>> ---
>> fs/ext4/super.c | 37 ++++++++++++++++++++++++++++++-------
>> fs/quota/dquot.c | 16 +++++++++++++---
>> include/linux/quotaops.h | 1 +
>> 3 files changed, 44 insertions(+), 10 deletions(-)
>>
>> diff --git a/fs/ext4/super.c b/fs/ext4/super.c
>> index 3929091..164217e 100644
>> --- a/fs/ext4/super.c
>> +++ b/fs/ext4/super.c
>> @@ -3777,14 +3777,37 @@ static int ext4_release_dquot(struct dquot *dquot)
>>
>> static int ext4_mark_dquot_dirty(struct dquot *dquot)
>> {
>> - /* Are we journaling quotas? */
>> - if (EXT4_SB(dquot->dq_sb)->s_qf_names[USRQUOTA] ||
>> - EXT4_SB(dquot->dq_sb)->s_qf_names[GRPQUOTA]) {
>> - dquot_mark_dquot_dirty(dquot);
>> - return ext4_write_dquot(dquot);
>> - } else {
>> + int ret, err;
>> + handle_t *handle;
>> + struct inode *inode;
>> +
>> + /* Are we not journaling quotas? */
>> + if (!EXT4_SB(dquot->dq_sb)->s_qf_names[USRQUOTA] &&
>> + !EXT4_SB(dquot->dq_sb)->s_qf_names[GRPQUOTA])
>> return dquot_mark_dquot_dirty(dquot);
>> - }
>> +
>> + /* journaling quotas case */
>> + inode = dquot_to_inode(dquot);
>> + handle = ext4_journal_start(inode,
>> + EXT4_QUOTA_TRANS_BLOCKS(dquot->dq_sb));
>> + if (IS_ERR(handle))
>> + return PTR_ERR(handle);
>> + if (!__dquot_mark_dquot_dirty(dquot))
>> + ret = dquot_commit(dquot);
>> + else
>> + /*
>> + * Dquot was already dirty. This means that other task already
>> + * started a transaction but not clear dirty bit yet (see
>> + * dquot_commit). Since the only one running transaction is
>> + * possible at a time. Then that task belongs to the same
>> + * transaction. We don'n have to actually write dquot changes
>> + * because that task will write it for for us.
>> + */
>> + ret = 0;
>> + err = ext4_journal_stop(handle);
>> + if (!ret)
>> + ret = err;
>> + return ret;
>> }
>>
>> static int ext4_write_info(struct super_block *sb, int type)
>> diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
>> index 6979224..e03eea0 100644
>> --- a/fs/quota/dquot.c
>> +++ b/fs/quota/dquot.c
>> @@ -311,14 +311,24 @@ static inline int mark_dquot_dirty(struct dquot *dquot)
>> {
>> return dquot->dq_sb->dq_op->mark_dirty(dquot);
>> }
>> -
>> -int dquot_mark_dquot_dirty(struct dquot *dquot)
>> +/* mark dquot dirty in atomic meaner, and return old dirty state */
>> +int __dquot_mark_dquot_dirty(struct dquot *dquot)
>> {
>> + int ret = 1;
>> spin_lock(&dq_list_lock);
>> - if (!test_and_set_bit(DQ_MOD_B, &dquot->dq_flags))
>> + if (!test_and_set_bit(DQ_MOD_B, &dquot->dq_flags)) {
>> list_add(&dquot->dq_dirty, &sb_dqopt(dquot->dq_sb)->
>> info[dquot->dq_type].dqi_dirty_list);
>> + ret = 0;
>> + }
>> spin_unlock(&dq_list_lock);
>> + return ret;
>> +}
>> +EXPORT_SYMBOL(__dquot_mark_dquot_dirty);
>> +
>> +int dquot_mark_dquot_dirty(struct dquot *dquot)
>> +{
>> + __dquot_mark_dquot_dirty(dquot);
>> return 0;
>> }
>> EXPORT_SYMBOL(dquot_mark_dquot_dirty);
>> diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
>> index ae7ef25..c950f7d 100644
>> --- a/include/linux/quotaops.h
>> +++ b/include/linux/quotaops.h
>> @@ -56,6 +56,7 @@ int dquot_commit(struct dquot *dquot);
>> int dquot_acquire(struct dquot *dquot);
>> int dquot_release(struct dquot *dquot);
>> int dquot_commit_info(struct super_block *sb, int type);
>> +int __dquot_mark_dquot_dirty(struct dquot *dquot);
>> int dquot_mark_dquot_dirty(struct dquot *dquot);
>>
>> int vfs_quota_on(struct super_block *sb, int type, int format_id,
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] ext4: Update documentation to correct the inode_readahead_blks option name

2009-12-16T23:40:23Z

Per commit 240799cd, the option name for readahead should be
inode_readahead_blks, not inode_readahead.

Signed-off-by: Fang Wenqi <antonf@...>
---
Documentation/filesystems/ext4.txt | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Documentation/filesystems/ext4.txt b/Documentation/filesystems/ext4.txt
index af6885c..e1def17 100644
--- a/Documentation/filesystems/ext4.txt
+++ b/Documentation/filesystems/ext4.txt
@@ -196,7 +196,7 @@ nobarrier This also requires an IO stack which can support
also be used to enable or disable barriers, for
consistency with other ext4 mount options.

-inode_readahead=n This tuning parameter controls the maximum
+inode_readahead_blks=n This tuning parameter controls the maximum
number of inode table blocks that ext4's inode
table readahead algorithm will pre-read into
the buffer cache. The default value is 32 blocks.
--
1.6.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: journalled quota seed-up

2009-12-16T17:33:35Z

Dmitry Monakhov <dmonakhov@...> writes:

Sorry forgot to add linux-ext4@... list to CC

> Currently each quota modification result in write_dquot()
> and later dquot_commit(). This means what each quota modification
> function must wait for dqio_mutex. Which is *huge* performance
> penalty on big SMP systems. ASAIU The core idea of this implementation
> is to guarantee that each quota modification will be written to journal
> atomically. But in fact this is not always true, because dquot may be
> changed after dquot modification, but before it was committed to disk.
>
> | Task 1 | Task 2 |
> | alloc_space(nr) | |
> | ->spin_lock(dq_data_lock) | |
> | ->curspace += nr | |
> | ->spin_unlock(dq_data_lock) | |
> | ->mark_dirty() | free_space(nr) |
> | -->write_dquot() | ->spin_lock(dq_data_lock) |
> | --->dquot_commit() | ->curspace -= nr |
> | ---->commit_dqblk() | ->spin_unlock(dq_data_lock) |
> | ----->spin_lock(dq_data_lock) | |
> | ----->mem2disk_dqblk(ddq, dquot) | <<< Copy updated value |
> | ----->spin_unlock(dq_data_lock) | |
> | ----->quota_write() | |
> Quota corruption not happens only because quota modification caller
> started journal already. And ext3/4 allow only one running quota
> at a time. Let's exploit this fact and avoid writing quota each time.
> Act similar to dirty_for_io in general write_back path in page-cache.
> If we have found what other task already started on copy and write the
> dquot then we skip actual quota_write stage. And let that task do the job.
> Side effect: Task which skip real quota_write() will not get an error
> (if any). But this is not big deal because:
> 1) Any error has global consequences (RO_remount, err_print, etc).
> 2) Real IO is differed till the journall_commit.
>
> Signed-off-by: Dmitry Monakhov <dmonakhov@...>
> ---
> fs/ext4/super.c | 37 ++++++++++++++++++++++++++++++-------
> fs/quota/dquot.c | 16 +++++++++++++---
> include/linux/quotaops.h | 1 +
> 3 files changed, 44 insertions(+), 10 deletions(-)
>
> diff --git a/fs/ext4/super.c b/fs/ext4/super.c
> index 3929091..164217e 100644
> --- a/fs/ext4/super.c
> +++ b/fs/ext4/super.c
> @@ -3777,14 +3777,37 @@ static int ext4_release_dquot(struct dquot *dquot)
>
> static int ext4_mark_dquot_dirty(struct dquot *dquot)
> {
> - /* Are we journaling quotas? */
> - if (EXT4_SB(dquot->dq_sb)->s_qf_names[USRQUOTA] ||
> - EXT4_SB(dquot->dq_sb)->s_qf_names[GRPQUOTA]) {
> - dquot_mark_dquot_dirty(dquot);
> - return ext4_write_dquot(dquot);
> - } else {
> + int ret, err;
> + handle_t *handle;
> + struct inode *inode;
> +
> + /* Are we not journaling quotas? */
> + if (!EXT4_SB(dquot->dq_sb)->s_qf_names[USRQUOTA] &&
> + !EXT4_SB(dquot->dq_sb)->s_qf_names[GRPQUOTA])
> return dquot_mark_dquot_dirty(dquot);
> - }
> +
> + /* journaling quotas case */
> + inode = dquot_to_inode(dquot);
> + handle = ext4_journal_start(inode,
> + EXT4_QUOTA_TRANS_BLOCKS(dquot->dq_sb));
> + if (IS_ERR(handle))
> + return PTR_ERR(handle);
> + if (!__dquot_mark_dquot_dirty(dquot))
> + ret = dquot_commit(dquot);
> + else
> + /*
> + * Dquot was already dirty. This means that other task already
> + * started a transaction but not clear dirty bit yet (see
> + * dquot_commit). Since the only one running transaction is
> + * possible at a time. Then that task belongs to the same
> + * transaction. We don'n have to actually write dquot changes
> + * because that task will write it for for us.
> + */
> + ret = 0;
> + err = ext4_journal_stop(handle);
> + if (!ret)
> + ret = err;
> + return ret;
> }
>
> static int ext4_write_info(struct super_block *sb, int type)
> diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
> index 6979224..e03eea0 100644
> --- a/fs/quota/dquot.c
> +++ b/fs/quota/dquot.c
> @@ -311,14 +311,24 @@ static inline int mark_dquot_dirty(struct dquot *dquot)
> {
> return dquot->dq_sb->dq_op->mark_dirty(dquot);
> }
> -
> -int dquot_mark_dquot_dirty(struct dquot *dquot)
> +/* mark dquot dirty in atomic meaner, and return old dirty state */
> +int __dquot_mark_dquot_dirty(struct dquot *dquot)
> {
> + int ret = 1;
> spin_lock(&dq_list_lock);
> - if (!test_and_set_bit(DQ_MOD_B, &dquot->dq_flags))
> + if (!test_and_set_bit(DQ_MOD_B, &dquot->dq_flags)) {
> list_add(&dquot->dq_dirty, &sb_dqopt(dquot->dq_sb)->
> info[dquot->dq_type].dqi_dirty_list);
> + ret = 0;
> + }
> spin_unlock(&dq_list_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL(__dquot_mark_dquot_dirty);
> +
> +int dquot_mark_dquot_dirty(struct dquot *dquot)
> +{
> + __dquot_mark_dquot_dirty(dquot);
> return 0;
> }
> EXPORT_SYMBOL(dquot_mark_dquot_dirty);
> diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
> index ae7ef25..c950f7d 100644
> --- a/include/linux/quotaops.h
> +++ b/include/linux/quotaops.h
> @@ -56,6 +56,7 @@ int dquot_commit(struct dquot *dquot);
> int dquot_acquire(struct dquot *dquot);
> int dquot_release(struct dquot *dquot);
> int dquot_commit_info(struct super_block *sb, int type);
> +int __dquot_mark_dquot_dirty(struct dquot *dquot);
> int dquot_mark_dquot_dirty(struct dquot *dquot);
>
> int vfs_quota_on(struct super_block *sb, int type, int format_id,

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] ext4: flush delalloc blocks when space is low

2009-12-16T12:46:02Z

> Creating many small files in rapid succession on a small
> filesystem can lead to spurious ENOSPC; on a 104MB filesystem:
>
> for i in `seq 1 22500`; do
> echo -n > $SCRATCH_MNT/$i
> echo XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > $SCRATCH_MNT/$i
> done
>
> leads to ENOSPC even though after a sync, 40% of the fs is free
> again.
>
> This is because we reserve worst-case metadata for delalloc writes,
> and when data is allocated that worst-case reservation is not
> usually needed.
>
> When freespace is low, kicking off an async writeback will start
> converting that worst-case space usage into something more realistic,
> almost always freeing up space to continue.
>
> This resolves the testcase for me, and survives all 4 generic
> ENOSPC tests in xfstests.
>
> We'll still need a hard synchronous sync to squeeze out the last bit,
> but this fixes things up to a large degree.
>
> Signed-off-by: Eric Sandeen <sandeen@...>

Looks good.

Acked-by: Jan Kara <jack@...>

Honza

> ---
>
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 5c5bc5d..5b3f468 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -3024,11 +3024,18 @@ static int ext4_nonda_switch(struct super_block *sb)
> if (2 * free_blocks < 3 * dirty_blocks ||
> free_blocks < (dirty_blocks + EXT4_FREEBLOCKS_WATERMARK)) {
> /*
> - * free block count is less that 150% of dirty blocks
> - * or free blocks is less that watermark
> + * free block count is less than 150% of dirty blocks
> + * or free blocks is less than watermark
> */
> return 1;
> }
> + /*
> + * Even if we don't switch but are nearing capacity,
> + * start pushing delalloc when 1/2 of free blocks are dirty.
> + */
> + if (free_blocks < 2 * dirty_blocks)
> + writeback_inodes_sb_if_idle(sb);
> +
> return 0;
> }
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Jan Kara <jack@...>
SuSE CR Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] fs-writeback: Add helper function to start writeback if idle

2009-12-16T12:44:09Z

> ext4, at least, would like to start pushing on writeback if it starts
> to get close to ENOSPC when reserving worst-case blocks for delalloc
> writes. Writing out delalloc data will convert those worst-case
> predictions into usually smaller actual usage, freeing up space
> before we hit ENOSPC based on this speculation.
>
> Thanks to Jens for the suggestion for the helper function,
> & the naming help.
>
> I've made the helper return status on whether writeback was
> started even though I don't plan to use it in the ext4 patch;
> it seems like it would be potentially useful to test this
> in some cases.
>
> Signed-off-by: Eric Sandeen <sandeen@...>

The patch looks good.
Acked-by: Jan Kara <jack@...>

Honza

> ---
>
> diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
> index 9d5360c..bff5f77 100644
> --- a/fs/fs-writeback.c
> +++ b/fs/fs-writeback.c
> @@ -1213,6 +1213,23 @@ void writeback_inodes_sb(struct super_block *sb)
> EXPORT_SYMBOL(writeback_inodes_sb);
>
> /**
> + * writeback_inodes_sb_if_idle - start writeback if none underway
> + * @sb: the superblock
> + *
> + * Invoke writeback_inodes_sb if no writeback is currently underway.
> + * Returns 1 if writeback was started, 0 if not.
> + */
> +int writeback_inodes_sb_if_idle(struct super_block *sb)
> +{
> + if (!writeback_in_progress(sb->s_bdi)) {
> + writeback_inodes_sb(sb);
> + return 1;
> + } else
> + return 0;
> +}
> +EXPORT_SYMBOL(writeback_inodes_sb_if_idle);
> +
> +/**
> * sync_inodes_sb - sync sb inode pages
> * @sb: the superblock
> *
> diff --git a/include/linux/writeback.h b/include/linux/writeback.h
> index 66ebddc..dc52482 100644
> --- a/include/linux/writeback.h
> +++ b/include/linux/writeback.h
> @@ -69,6 +69,7 @@ struct writeback_control {
> struct bdi_writeback;
> int inode_wait(void *);
> void writeback_inodes_sb(struct super_block *);
> +int writeback_inodes_sb_if_idle(struct super_block *);
> void sync_inodes_sb(struct super_block *);
> void writeback_inodes_wbc(struct writeback_control *wbc);
> long wb_do_writeback(struct bdi_writeback *wb, int force_wait);
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Potential data consistency issue with ASYNC_COMMIT feature

2009-12-16T12:33:03Z

> However, journal checksums failed horribly when we tried to enable
> them by default during the last merge window, because of bugs in ext4
> where we were modifying certain metadata blocks (in particular
> superblock and xattr's) without journalling them. (Note to self; we
> need to back port those fixes to ext3; the lack of journalling in
> xattr in particular could mean that in some cases we could lose some
> updates that could affect SELINUX after a crash.)
Already done by Eric and merged ;): d965736b8cb42ae51ba9c3f13488035a98d025c6,
b918397542388de75bd86c32fbfa820e5d629fa9.

Honza
--
Jan Kara <jack@...>
SuSE CR Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Confirmation ReferenceNumber:UK/2551256008/07

2009-12-16T07:31:01Z

Confirmation ReferenceNumber:UK/2551256008/07
You have won £850,000.00 in free lotto email program held 9th
December 2009 in In Liverpool.
Email Ticket Number X-3007893284-Z ReferenceNumber:UK/2551256008/07
Draw Lucky Number:09-22-37-39-41-49-2-7
BatchNumber:JPK14/0115/IP,Serial Number: HMML3214-07
Please note that the validity period of the winning is 24th of
December 2009.
Contact: Mr. Frank Arthur with your Full Name, Home Address, Age,Gender,
Occupation,Country Of Residence,Nationality & Telephone Number.
Email: claims-office@...<mailto:claims-office@...>
Tel: +4470111 83202+, 4470111 83206
+447031994752, +447031978650
NB Please Note do mail back through the alert email contact Mr. Frank Arthur
via email:claims-office@...
Sincerely,
Brooks, Ken
For The British National Lottery
**********************************************************************
This message is intended for the addressee named and may contain
privileged information or confidential information or both. If you
are not the intended recipient please delete it and notify the sender.
**********************************************************************
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[RFC PATCH 4/4] ext4: ext4_get_block_write and io_end code cleanup

2009-12-15T17:41:38Z

ext4: ext4_get_block_write and io_end code cleanup

Move ext4_get_block_write and io_end related code forward to get rid
of function declearation.

Signed-off-by: Jiaying Zhang <jiayingz@...>
---
fs/ext4/inode.c | 2179 +++++++++++++++++++++++++++-----------------------------
1 file changed, 1087 insertions(+), 1092 deletions(-)

Index: git-ext4/fs/ext4/inode.c
===================================================================
--- git-ext4.orig/fs/ext4/inode.c 2009-12-15 16:59:06.000000000 -0800
+++ git-ext4/fs/ext4/inode.c 2009-12-15 17:02:13.000000000 -0800
@@ -1493,7 +1493,47 @@ static int do_journal_get_write_access(h
}

static int ext4_get_block_write(struct inode *inode, sector_t iblock,
- struct buffer_head *bh_result, int create);
+ struct buffer_head *bh_result, int create)
+{
+ handle_t *handle = ext4_journal_current_handle();
+ int ret = 0;
+ unsigned max_blocks = bh_result->b_size >> inode->i_blkbits;
+ int dio_credits;
+ int started = 0;
+
+ ext4_debug("ext4_get_block_write: inode %lu, create flag %d\n",
+ inode->i_ino, create);
+ /*
+ * ext4_get_block in prepare for a DIO write or buffer write.
+ * We allocate an uinitialized extent if blocks haven't been allocated.
+ * The extent will be converted to initialized after IO complete.
+ */
+ create = EXT4_GET_BLOCKS_IO_CREATE_EXT;
+
+ if (!handle) {
+ if (max_blocks > DIO_MAX_BLOCKS)
+ max_blocks = DIO_MAX_BLOCKS;
+ dio_credits = ext4_chunk_trans_blocks(inode, max_blocks);
+ handle = ext4_journal_start(inode, dio_credits);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ started = 1;
+ }
+
+ ret = ext4_get_blocks(handle, inode, iblock, max_blocks, bh_result,
+ create);
+ if (ret > 0) {
+ bh_result->b_size = (ret << inode->i_blkbits);
+ ret = 0;
+ }
+ if (started)
+ ext4_journal_stop(handle);
+out:
+ return ret;
+}
+
static int ext4_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata)
@@ -2607,746 +2647,497 @@ out:
return ret;
}

-static int ext4_set_bh_endio(struct buffer_head *bh, struct inode *inode);
-static void ext4_end_io_buffer_write(struct buffer_head *bh, int uptodate);
+static void ext4_free_io_end(ext4_io_end_t *io)
+{
+ BUG_ON(!io);
+ iput(io->inode);
+ kfree(io);
+}
+
+static void dump_completed_IO(struct inode * inode)
+{
+#ifdef EXT4_DEBUG
+ struct list_head *cur, *before, *after;
+ ext4_io_end_t *io, *io0, *io1;
+
+ if (list_empty(&EXT4_I(inode)->i_completed_io_list)){
+ ext4_debug("inode %lu completed_io list is empty\n",
inode->i_ino);
+ return;
+ }
+
+ ext4_debug("Dump inode %lu completed_io list \n", inode->i_ino);
+ list_for_each_entry(io, &EXT4_I(inode)->i_completed_io_list, list){
+ cur = &io->list;
+ before = cur->prev;
+ io0 = container_of(before, ext4_io_end_t, list);
+ after = cur->next;
+ io1 = container_of(after, ext4_io_end_t, list);
+
+ ext4_debug("io 0x%p from inode %lu,prev 0x%p,next 0x%p\n",
+ io, inode->i_ino, io0, io1);
+ }
+#endif
+}

/*
- * Note that we don't need to start a transaction unless we're journaling data
- * because we should have holes filled from ext4_page_mkwrite(). We even don't
- * need to file the inode to the transaction's list in ordered mode because if
- * we are writing back data added by write(), the inode is already there and if
- * we are writing back data modified via mmap(), noone guarantees in which
- * transaction the data will hit the disk. In case we are journaling data, we
- * cannot start transaction directly because transaction start ranks above page
- * lock so we have to do some magic.
- *
- * This function can get called via...
- * - ext4_da_writepages after taking page lock (have journal handle)
- * - journal_submit_inode_data_buffers (no journal handle)
- * - shrink_page_list via pdflush (no journal handle)
- * - grab_page_cache when doing write_begin (have journal handle)
- *
- * We don't do any block allocation in this function. If we have page with
- * multiple blocks we need to write those buffer_heads that are mapped. This
- * is important for mmaped based write. So if we do with blocksize 1K
- * truncate(f, 1024);
- * a = mmap(f, 0, 4096);
- * a[0] = 'a';
- * truncate(f, 4096);
- * we have in the page first buffer_head mapped via page_mkwrite call back
- * but other bufer_heads would be unmapped but dirty(dirty done via the
- * do_wp_page). So writepage should write the first block. If we modify
- * the mmap area beyond 1024 we will again get a page_fault and the
- * page_mkwrite callback will do the block allocation and mark the
- * buffer_heads mapped.
- *
- * We redirty the page if we have any buffer_heads that is either delay or
- * unwritten in the page.
- *
- * We can get recursively called as show below.
- *
- * ext4_writepage() -> kmalloc() -> __alloc_pages() -> page_launder() ->
- * ext4_writepage()
- *
- * But since we don't do any block allocation we should not deadlock.
- * Page also have the dirty flag cleared so we don't get recurive page_lock.
+ * check a range of space and convert unwritten extents to written.
*/
-static int ext4_writepage(struct page *page,
- struct writeback_control *wbc)
+static int ext4_end_io_nolock(ext4_io_end_t *io)
{
+ struct inode *inode = io->inode;
+ loff_t offset = io->offset;
+ size_t size = io->size;
int ret = 0;
- loff_t size;
- unsigned int len;
- struct buffer_head *page_bufs = NULL;
- struct inode *inode = page->mapping->host;

- trace_ext4_writepage(inode, page);
- size = i_size_read(inode);
- if (page->index == size >> PAGE_CACHE_SHIFT)
- len = size & ~PAGE_CACHE_MASK;
- else
- len = PAGE_CACHE_SIZE;
+ ext4_debug("ext4_end_io_nolock: io 0x%p from inode %lu,list->next 0x%p,"
+ "list->prev 0x%p\n",
+ io, inode->i_ino, io->list.next, io->list.prev);

- if (page_has_buffers(page)) {
- page_bufs = page_buffers(page);
- if (walk_page_buffers(NULL, page_bufs, 0, len, NULL,
- ext4_bh_delay_or_unwritten)) {
- /*
- * We don't want to do block allocation
- * So redirty the page and return
- * We may reach here when we do a journal commit
- * via journal_submit_inode_data_buffers.
- * If we don't have mapping block we just ignore
- * them. We can also reach here via shrink_page_list
- */
- redirty_page_for_writepage(wbc, page);
- unlock_page(page);
- return 0;
- }
- } else {
- /*
- * The test for page_has_buffers() is subtle:
- * We know the page is dirty but it lost buffers. That means
- * that at some moment in time after write_begin()/write_end()
- * has been called all buffers have been clean and thus they
- * must have been written at least once. So they are all
- * mapped and we can happily proceed with mapping them
- * and writing the page.
- *
- * Try to initialize the buffer_heads and check whether
- * all are mapped and non delay. We don't want to
- * do block allocation here.
- */
- ret = block_prepare_write(page, 0, len,
- noalloc_get_block_write);
- if (!ret) {
- page_bufs = page_buffers(page);
- /* check whether all are mapped and non delay */
- if (walk_page_buffers(NULL, page_bufs, 0, len, NULL,
- ext4_bh_delay_or_unwritten)) {
- redirty_page_for_writepage(wbc, page);
- unlock_page(page);
- return 0;
- }
- } else {
- /*
- * We can't do block allocation here
- * so just redity the page and unlock
- * and return
- */
- redirty_page_for_writepage(wbc, page);
- unlock_page(page);
- return 0;
- }
- /* now mark the buffer_heads as dirty and uptodate */
- block_commit_write(page, 0, len);
- }
+ if (list_empty(&io->list))
+ return ret;

- if (PageChecked(page) && ext4_should_journal_data(inode)) {
- /*
- * It's mmapped pagecache. Add buffers and journal it. There
- * doesn't seem much point in redirtying the page here.
- */
- ClearPageChecked(page);
- return __ext4_journalled_writepage(page, len);
- }
+ if (io->flag != EXT4_IO_WRITTEN)
+ return ret;

- if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
- ret = nobh_writepage(page, noalloc_get_block_write, wbc);
- else if (page_bufs && buffer_uninit(page_bufs)) {
- ext4_set_bh_endio(page_bufs, inode);
- ret = block_write_full_page_endio(page, noalloc_get_block_write,
- wbc, ext4_end_io_buffer_write);
- } else
- ret = block_write_full_page(page, noalloc_get_block_write,
- wbc);
+ ret = ext4_convert_unwritten_extents(inode, offset, size);
+ if (ret < 0) {
+ printk(KERN_EMERG "%s: failed to convert unwritten"
+ "extents to written extents, error is %d"
+ " io is still on inode %lu aio dio list\n",
+ __func__, ret, inode->i_ino);
+ return ret;
+ }

+ /* clear the DIO AIO unwritten flag */
+ io->flag = 0;
return ret;
}

/*
- * This is called via ext4_da_writepages() to
- * calulate the total number of credits to reserve to fit
- * a single extent allocation into a single transaction,
- * ext4_da_writpeages() will loop calling this before
- * the block allocation.
+ * work on completed aio dio IO, to convert unwritten extents to extents
*/
-
-static int ext4_da_writepages_trans_blocks(struct inode *inode)
+static void ext4_end_io_work(struct work_struct *work)
{
- int max_blocks = EXT4_I(inode)->i_reserved_data_blocks;
-
- /*
- * With non-extent format the journal credit needed to
- * insert nrblocks contiguous block is dependent on
- * number of contiguous block. So we will limit
- * number of contiguous block to a sane value
- */
- if (!(EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL) &&
- (max_blocks > EXT4_MAX_TRANS_DATA))
- max_blocks = EXT4_MAX_TRANS_DATA;
+ ext4_io_end_t *io = container_of(work, ext4_io_end_t, work);
+ struct inode *inode = io->inode;
+ int ret = 0;

- return ext4_chunk_trans_blocks(inode, max_blocks);
+ mutex_lock(&inode->i_mutex);
+ ret = ext4_end_io_nolock(io);
+ if (ret >= 0) {
+ if (!list_empty(&io->list))
+ list_del_init(&io->list);
+ ext4_free_io_end(io);
+ }
+ mutex_unlock(&inode->i_mutex);
}

-static int ext4_da_writepages(struct address_space *mapping,
- struct writeback_control *wbc)
-{
- pgoff_t index;
- int range_whole = 0;
- handle_t *handle = NULL;
- struct mpage_da_data mpd;
- struct inode *inode = mapping->host;
- int no_nrwrite_index_update;
- int pages_written = 0;
- long pages_skipped;
- unsigned int max_pages;
- int range_cyclic, cycled = 1, io_done = 0;
- int needed_blocks, ret = 0;
- long desired_nr_to_write, nr_to_writebump = 0;
- loff_t range_start = wbc->range_start;
- struct ext4_sb_info *sbi = EXT4_SB(mapping->host->i_sb);
-
- trace_ext4_da_writepages(inode, wbc);
-
- /*
- * No pages to write? This is mainly a kludge to avoid starting
- * a transaction for special inodes like journal inode on last iput()
- * because that could violate lock ordering on umount
- */
- if (!mapping->nrpages || !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
- return 0;
-
- /*
- * If the filesystem has aborted, it is read-only, so return
- * right away instead of dumping stack traces later on that
- * will obscure the real source of the problem. We test
- * EXT4_MF_FS_ABORTED instead of sb->s_flag's MS_RDONLY because
- * the latter could be true if the filesystem is mounted
- * read-only, and in that case, ext4_da_writepages should
- * *never* be called, so if that ever happens, we would want
- * the stack trace.
- */
- if (unlikely(sbi->s_mount_flags & EXT4_MF_FS_ABORTED))
- return -EROFS;
-
- if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
- range_whole = 1;
-
- range_cyclic = wbc->range_cyclic;
- if (wbc->range_cyclic) {
- index = mapping->writeback_index;
- if (index)
- cycled = 0;
- wbc->range_start = index << PAGE_CACHE_SHIFT;
- wbc->range_end = LLONG_MAX;
- wbc->range_cyclic = 0;
- } else
- index = wbc->range_start >> PAGE_CACHE_SHIFT;
-
- /*
- * This works around two forms of stupidity. The first is in
- * the writeback code, which caps the maximum number of pages
- * written to be 1024 pages. This is wrong on multiple
- * levels; different architectues have a different page size,
- * which changes the maximum amount of data which gets
- * written. Secondly, 4 megabytes is way too small. XFS
- * forces this value to be 16 megabytes by multiplying
- * nr_to_write parameter by four, and then relies on its
- * allocator to allocate larger extents to make them
- * contiguous. Unfortunately this brings us to the second
- * stupidity, which is that ext4's mballoc code only allocates
- * at most 2048 blocks. So we force contiguous writes up to
- * the number of dirty blocks in the inode, or
- * sbi->max_writeback_mb_bump whichever is smaller.
- */
- max_pages = sbi->s_max_writeback_mb_bump << (20 - PAGE_CACHE_SHIFT);
- if (!range_cyclic && range_whole)
- desired_nr_to_write = wbc->nr_to_write * 8;
- else
- desired_nr_to_write = ext4_num_dirty_pages(inode, index,
- max_pages);
- if (desired_nr_to_write > max_pages)
- desired_nr_to_write = max_pages;
-
- if (wbc->nr_to_write < desired_nr_to_write) {
- nr_to_writebump = desired_nr_to_write - wbc->nr_to_write;
- wbc->nr_to_write = desired_nr_to_write;
- }
-
- mpd.wbc = wbc;
- mpd.inode = mapping->host;
-
- /*
- * we don't want write_cache_pages to update
- * nr_to_write and writeback_index
- */
- no_nrwrite_index_update = wbc->no_nrwrite_index_update;
- wbc->no_nrwrite_index_update = 1;
- pages_skipped = wbc->pages_skipped;
-
-retry:
- while (!ret && wbc->nr_to_write > 0) {
-
- /*
- * we insert one extent at a time. So we need
- * credit needed for single extent allocation.
- * journalled mode is currently not supported
- * by delalloc
- */
- BUG_ON(ext4_should_journal_data(inode));
- needed_blocks = ext4_da_writepages_trans_blocks(inode);
-
- /* start a new transaction*/
- handle = ext4_journal_start(inode, needed_blocks);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- ext4_msg(inode->i_sb, KERN_CRIT, "%s: jbd2_start: "
- "%ld pages, ino %lu; err %d\n", __func__,
- wbc->nr_to_write, inode->i_ino, ret);
- goto out_writepages;
- }
-
- /*
- * Now call __mpage_da_writepage to find the next
- * contiguous region of logical blocks that need
- * blocks to be allocated by ext4. We don't actually
- * submit the blocks for I/O here, even though
- * write_cache_pages thinks it will, and will set the
- * pages as clean for write before calling
- * __mpage_da_writepage().
- */
- mpd.b_size = 0;
- mpd.b_state = 0;
- mpd.b_blocknr = 0;
- mpd.first_page = 0;
- mpd.next_page = 0;
- mpd.io_done = 0;
- mpd.pages_written = 0;
- mpd.retval = 0;
- ret = write_cache_pages(mapping, wbc, __mpage_da_writepage,
- &mpd);
- /*
- * If we have a contigous extent of pages and we
- * haven't done the I/O yet, map the blocks and submit
- * them for I/O.
- */
- if (!mpd.io_done && mpd.next_page != mpd.first_page) {
- if (mpage_da_map_blocks(&mpd) == 0)
- mpage_da_submit_io(&mpd);
- mpd.io_done = 1;
- ret = MPAGE_DA_EXTENT_TAIL;
- }
- trace_ext4_da_write_pages(inode, &mpd);
- wbc->nr_to_write -= mpd.pages_written;
-
- ext4_journal_stop(handle);
-
- if ((mpd.retval == -ENOSPC) && sbi->s_journal) {
- /* commit the transaction which would
- * free blocks released in the transaction
- * and try again
- */
- jbd2_journal_force_commit_nested(sbi->s_journal);
- wbc->pages_skipped = pages_skipped;
- ret = 0;
- } else if (ret == MPAGE_DA_EXTENT_TAIL) {
- /*
- * got one extent now try with
- * rest of the pages
- */
- pages_written += mpd.pages_written;
- wbc->pages_skipped = pages_skipped;
- ret = 0;
- io_done = 1;
- } else if (wbc->nr_to_write)
- /*
- * There is no more writeout needed
- * or we requested for a noblocking writeout
- * and we found the device congested
- */
- break;
- }
- if (!io_done && !cycled) {
- cycled = 1;
- index = 0;
- wbc->range_start = index << PAGE_CACHE_SHIFT;
- wbc->range_end = mapping->writeback_index - 1;
- goto retry;
- }
- if (pages_skipped != wbc->pages_skipped)
- ext4_msg(inode->i_sb, KERN_CRIT,
- "This should not happen leaving %s "
- "with nr_to_write = %ld ret = %d\n",
- __func__, wbc->nr_to_write, ret);
-
- /* Update index */
- index += pages_written;
- wbc->range_cyclic = range_cyclic;
- if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0))
- /*
- * set the writeback_index so that range_cyclic
- * mode will write it back later
- */
- mapping->writeback_index = index;
-
-out_writepages:
- if (!no_nrwrite_index_update)
- wbc->no_nrwrite_index_update = 0;
- if (wbc->nr_to_write > nr_to_writebump)
- wbc->nr_to_write -= nr_to_writebump;
- wbc->range_start = range_start;
- trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
- return ret;
-}
-
-#define FALL_BACK_TO_NONDELALLOC 1
-static int ext4_nonda_switch(struct super_block *sb)
-{
- s64 free_blocks, dirty_blocks;
- struct ext4_sb_info *sbi = EXT4_SB(sb);
-
- /*
- * switch to non delalloc mode if we are running low
- * on free block. The free block accounting via percpu
- * counters can get slightly wrong with percpu_counter_batch getting
- * accumulated on each CPU without updating global counters
- * Delalloc need an accurate free block accounting. So switch
- * to non delalloc when we are near to error range.
- */
- free_blocks = percpu_counter_read_positive(&sbi->s_freeblocks_counter);
- dirty_blocks =
percpu_counter_read_positive(&sbi->s_dirtyblocks_counter);
- if (2 * free_blocks < 3 * dirty_blocks ||
- free_blocks < (dirty_blocks + EXT4_FREEBLOCKS_WATERMARK)) {
- /*
- * free block count is less that 150% of dirty blocks
- * or free blocks is less that watermark
- */
- return 1;
- }
- return 0;
-}
-
-static int ext4_da_write_begin(struct file *file, struct
address_space *mapping,
- loff_t pos, unsigned len, unsigned flags,
- struct page **pagep, void **fsdata)
+/*
+ * This function is called from ext4_sync_file().
+ *
+ * When IO is completed, the work to convert unwritten extents to
+ * written is queued on workqueue but may not get immediately
+ * scheduled. When fsync is called, we need to ensure the
+ * conversion is complete before fsync returns.
+ * The inode keeps track of a list of pending/completed IO that
+ * might needs to do the conversion. This function walks through
+ * the list and convert the related unwritten extents for completed IO
+ * to written.
+ * The function return the number of pending IOs on success.
+ */
+int flush_completed_IO(struct inode *inode)
{
- int ret, retries = 0;
- struct page *page;
- pgoff_t index;
- unsigned from, to;
- struct inode *inode = mapping->host;
- handle_t *handle;
-
- index = pos >> PAGE_CACHE_SHIFT;
- from = pos & (PAGE_CACHE_SIZE - 1);
- to = from + len;
-
- if (ext4_nonda_switch(inode->i_sb)) {
- *fsdata = (void *)FALL_BACK_TO_NONDELALLOC;
- return ext4_write_begin(file, mapping, pos,
- len, flags, pagep, fsdata);
- }
- *fsdata = (void *)0;
- trace_ext4_da_write_begin(inode, pos, len, flags);
-retry:
- /*
- * With delayed allocation, we don't log the i_disksize update
- * if there is delayed block allocation. But we still need
- * to journalling the i_disksize update if writes to the end
- * of file which has an already mapped buffer.
- */
- handle = ext4_journal_start(inode, 1);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
- }
- /* We cannot recurse into the filesystem as the transaction is already
- * started */
- flags |= AOP_FLAG_NOFS;
+ ext4_io_end_t *io, *tmp;
+ int ret = 0;
+ int ret2 = 0;

- page = grab_cache_page_write_begin(mapping, index, flags);
- if (!page) {
- ext4_journal_stop(handle);
- ret = -ENOMEM;
- goto out;
- }
- *pagep = page;
+ if (list_empty(&EXT4_I(inode)->i_completed_io_list))
+ return ret;

- ret = block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
- ext4_da_get_block_prep);
- if (ret < 0) {
- unlock_page(page);
- ext4_journal_stop(handle);
- page_cache_release(page);
+ dump_completed_IO(inode);
+ list_for_each_entry_safe(io, tmp,
+ &EXT4_I(inode)->i_completed_io_list, list) {
+ if (io->flag == EXT4_IO_UNWRITTEN)
+ continue;
/*
- * block_write_begin may have instantiated a few blocks
- * outside i_size. Trim these off again. Don't need
- * i_size_read because we hold i_mutex.
+ * Calling ext4_end_io_nolock() to convert completed
+ * IO to written.
+ *
+ * When ext4_sync_file() is called, run_queue() may already
+ * about to flush the work corresponding to this io structure.
+ * It will be upset if it founds the io structure related
+ * to the work-to-be schedule is freed.
+ *
+ * Thus we need to keep the io structure still valid here after
+ * convertion finished. The io structure has a flag to
+ * avoid double converting from both fsync and background work
+ * queue work.
*/
- if (pos + len > inode->i_size)
- ext4_truncate(inode);
+ ret = ext4_end_io_nolock(io);
+ if (ret < 0)
+ ret2 = ret;
+ else
+ list_del_init(&io->list);
}
-
- if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
- goto retry;
-out:
- return ret;
+ return (ret2 < 0) ? ret2 : 0;
}

-/*
- * Check if we should update i_disksize
- * when write to the end of file but not require block allocation
- */
-static int ext4_da_should_update_i_disksize(struct page *page,
- unsigned long offset)
+static ext4_io_end_t *ext4_init_io_end (struct inode *inode)
{
- struct buffer_head *bh;
- struct inode *inode = page->mapping->host;
- unsigned int idx;
- int i;
-
- bh = page_buffers(page);
- idx = offset >> inode->i_blkbits;
-
- for (i = 0; i < idx; i++)
- bh = bh->b_this_page;
-
- if (!buffer_mapped(bh) || (buffer_delay(bh)) || buffer_unwritten(bh))
- return 0;
- return 1;
-}
+ ext4_io_end_t *io = NULL;

-static int ext4_da_write_end(struct file *file,
- struct address_space *mapping,
- loff_t pos, unsigned len, unsigned copied,
- struct page *page, void *fsdata)
-{
- struct inode *inode = mapping->host;
- int ret = 0, ret2;
- handle_t *handle = ext4_journal_current_handle();
- loff_t new_i_size;
- unsigned long start, end;
- int write_mode = (int)(unsigned long)fsdata;
+ io = kmalloc(sizeof(*io), GFP_NOFS);

- if (write_mode == FALL_BACK_TO_NONDELALLOC) {
- if (ext4_should_order_data(inode)) {
- return ext4_ordered_write_end(file, mapping, pos,
- len, copied, page, fsdata);
- } else if (ext4_should_writeback_data(inode)) {
- return ext4_writeback_write_end(file, mapping, pos,
- len, copied, page, fsdata);
- } else {
- BUG();
- }
+ if (io) {
+ igrab(inode);
+ io->inode = inode;
+ io->flag = 0;
+ io->offset = 0;
+ io->size = 0;
+ io->error = 0;
+ INIT_WORK(&io->work, ext4_end_io_work);
+ INIT_LIST_HEAD(&io->list);
}

- trace_ext4_da_write_end(inode, pos, len, copied);
- start = pos & (PAGE_CACHE_SIZE - 1);
- end = start + copied - 1;
+ return io;
+}

- /*
- * generic_write_end() will run mark_inode_dirty() if i_size
- * changes. So let's piggyback the i_disksize mark_inode_dirty
- * into that.
- */
+static void ext4_end_io_dio(struct kiocb *iocb, loff_t offset,
+ ssize_t size, void *private)
+{
+ ext4_io_end_t *io_end = iocb->private;
+ struct workqueue_struct *wq;

- new_i_size = pos + copied;
- if (new_i_size > EXT4_I(inode)->i_disksize) {
- if (ext4_da_should_update_i_disksize(page, end)) {
- down_write(&EXT4_I(inode)->i_data_sem);
- if (new_i_size > EXT4_I(inode)->i_disksize) {
- /*
- * Updating i_disksize when extending file
- * without needing block allocation
- */
- if (ext4_should_order_data(inode))
- ret = ext4_jbd2_file_inode(handle,
- inode);
+ /* if not async direct IO or dio with 0 bytes write, just return */
+ if (!io_end || !size)
+ return;

- EXT4_I(inode)->i_disksize = new_i_size;
- }
- up_write(&EXT4_I(inode)->i_data_sem);
- /* We need to mark inode dirty even if
- * new_i_size is less that inode->i_size
- * bu greater than i_disksize.(hint delalloc)
- */
- ext4_mark_inode_dirty(handle, inode);
- }
+ ext_debug("ext4_end_io_dio(): io_end 0x%p"
+ "for inode %lu, iocb 0x%p, offset %llu, size %llu\n",
+ iocb->private, io_end->inode->i_ino, iocb, offset,
+ size);
+
+ /* if not aio dio with unwritten extents, just free io and return */
+ if (io_end->flag != EXT4_IO_UNWRITTEN){
+ ext4_free_io_end(io_end);
+ iocb->private = NULL;
+ return;
}
- ret2 = generic_write_end(file, mapping, pos, len, copied,
- page, fsdata);
- copied = ret2;
- if (ret2 < 0)
- ret = ret2;
- ret2 = ext4_journal_stop(handle);
- if (!ret)
- ret = ret2;

- return ret ? ret : copied;
+ io_end->offset = offset;
+ io_end->size = size;
+ io_end->flag = EXT4_IO_WRITTEN;
+ wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;
+
+ /* queue the work to convert unwritten extents to written */
+ queue_work(wq, &io_end->work);
+
+ /* Add the io_end to per-inode completed aio dio list*/
+ list_add_tail(&io_end->list,
+ &EXT4_I(io_end->inode)->i_completed_io_list);
+ iocb->private = NULL;
}

-static void ext4_da_invalidatepage(struct page *page, unsigned long offset)
+static void ext4_end_io_buffer_write(struct buffer_head *bh, int uptodate)
{
- /*
- * Drop reserved blocks
- */
- BUG_ON(!PageLocked(page));
- if (!page_has_buffers(page))
+ ext4_io_end_t *io_end = bh->b_private;
+ struct workqueue_struct *wq;
+
+ if (!io_end)
goto out;
+ io_end->flag = EXT4_IO_WRITTEN;
+ wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;
+ /* queue the work to convert unwritten extents to written */
+ queue_work(wq, &io_end->work);
+out:
+ bh->b_private = NULL;
+ bh->b_end_io = NULL;
+ clear_buffer_uninit(bh);
+ end_buffer_async_write(bh, uptodate);
+}

- ext4_da_page_release_reservation(page, offset);
+static int ext4_set_bh_endio(struct buffer_head *bh, struct inode *inode)
+{
+ ext4_io_end_t *io_end;
+ struct page *page = bh->b_page;
+ loff_t offset = (sector_t)page->index << PAGE_CACHE_SHIFT;
+ size_t size = bh->b_size;

-out:
- ext4_invalidatepage(page, offset);
+ io_end = ext4_init_io_end(inode);
+ if (!io_end)
+ return -ENOMEM;
+ io_end->offset = offset;
+ io_end->size = size;
+ io_end->flag = EXT4_IO_UNWRITTEN;
+ /* Add the io_end to per-inode completed io list*/
+ list_add_tail(&io_end->list,
+ &EXT4_I(io_end->inode)->i_completed_io_list);

- return;
+ bh->b_private = io_end;
+ bh->b_end_io = ext4_end_io_buffer_write;
+ return 0;
}

/*
- * Force all delayed allocation blocks to be allocated for a given inode.
+ * Note that we don't need to start a transaction unless we're journaling data
+ * because we should have holes filled from ext4_page_mkwrite(). We even don't
+ * need to file the inode to the transaction's list in ordered mode because if
+ * we are writing back data added by write(), the inode is already there and if
+ * we are writing back data modified via mmap(), noone guarantees in which
+ * transaction the data will hit the disk. In case we are journaling data, we
+ * cannot start transaction directly because transaction start ranks above page
+ * lock so we have to do some magic.
+ *
+ * This function can get called via...
+ * - ext4_da_writepages after taking page lock (have journal handle)
+ * - journal_submit_inode_data_buffers (no journal handle)
+ * - shrink_page_list via pdflush (no journal handle)
+ * - grab_page_cache when doing write_begin (have journal handle)
+ *
+ * We don't do any block allocation in this function. If we have page with
+ * multiple blocks we need to write those buffer_heads that are mapped. This
+ * is important for mmaped based write. So if we do with blocksize 1K
+ * truncate(f, 1024);
+ * a = mmap(f, 0, 4096);
+ * a[0] = 'a';
+ * truncate(f, 4096);
+ * we have in the page first buffer_head mapped via page_mkwrite call back
+ * but other bufer_heads would be unmapped but dirty(dirty done via the
+ * do_wp_page). So writepage should write the first block. If we modify
+ * the mmap area beyond 1024 we will again get a page_fault and the
+ * page_mkwrite callback will do the block allocation and mark the
+ * buffer_heads mapped.
+ *
+ * We redirty the page if we have any buffer_heads that is either delay or
+ * unwritten in the page.
+ *
+ * We can get recursively called as show below.
+ *
+ * ext4_writepage() -> kmalloc() -> __alloc_pages() -> page_launder() ->
+ * ext4_writepage()
+ *
+ * But since we don't do any block allocation we should not deadlock.
+ * Page also have the dirty flag cleared so we don't get recurive page_lock.
*/
-#if 1
-int ext4_alloc_da_blocks(struct inode *inode)
+static int ext4_writepage(struct page *page,
+ struct writeback_control *wbc)
{
- trace_ext4_alloc_da_blocks(inode);
-
- if (!EXT4_I(inode)->i_reserved_data_blocks &&
- !EXT4_I(inode)->i_reserved_meta_blocks)
- return 0;
+ int ret = 0;
+ loff_t size;
+ unsigned int len;
+ struct buffer_head *page_bufs = NULL;
+ struct inode *inode = page->mapping->host;

- /*
- * We do something simple for now. The filemap_flush() will
- * also start triggering a write of the data blocks, which is
- * not strictly speaking necessary (and for users of
- * laptop_mode, not even desirable). However, to do otherwise
- * would require replicating code paths in:
- *
- * ext4_da_writepages() ->
- * write_cache_pages() ---> (via passed in callback function)
- * __mpage_da_writepage() -->
- * mpage_add_bh_to_extent()
- * mpage_da_map_blocks()
- *
- * The problem is that write_cache_pages(), located in
- * mm/page-writeback.c, marks pages clean in preparation for
- * doing I/O, which is not desirable if we're not planning on
- * doing I/O at all.
- *
- * We could call write_cache_pages(), and then redirty all of
- * the pages by calling redirty_page_for_writeback() but that
- * would be ugly in the extreme. So instead we would need to
- * replicate parts of the code in the above functions,
- * simplifying them becuase we wouldn't actually intend to
- * write out the pages, but rather only collect contiguous
- * logical block extents, call the multi-block allocator, and
- * then update the buffer heads with the block allocations.
- *
- * For now, though, we'll cheat by calling filemap_flush(),
- * which will map the blocks, and start the I/O, but not
- * actually wait for the I/O to complete.
- */
- return filemap_flush(inode->i_mapping);
-}
-#else
-static int flush_alloc_da_page(struct page *page, struct mpage_da_data *mpd)
-{
- struct inode *inode = mpd->inode;
- struct buffer_head *bh, *head;
- sector_t logical;
+ trace_ext4_writepage(inode, page);
+ size = i_size_read(inode);
+ if (page->index == size >> PAGE_CACHE_SHIFT)
+ len = size & ~PAGE_CACHE_MASK;
+ else
+ len = PAGE_CACHE_SIZE;

- /*
- * Can we merge this page to current extent?
- */
- if (mpd->next_page != page->index) {
+ if (page_has_buffers(page)) {
+ page_bufs = page_buffers(page);
+ if (walk_page_buffers(NULL, page_bufs, 0, len, NULL,
+ ext4_bh_delay_or_unwritten)) {
+ /*
+ * We don't want to do block allocation
+ * So redirty the page and return
+ * We may reach here when we do a journal commit
+ * via journal_submit_inode_data_buffers.
+ * If we don't have mapping block we just ignore
+ * them. We can also reach here via shrink_page_list
+ */
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ return 0;
+ }
+ } else {
/*
- * Nope, we can't. So, we map non-allocated blocks
- * and start IO on them using writepage()
+ * The test for page_has_buffers() is subtle:
+ * We know the page is dirty but it lost buffers. That means
+ * that at some moment in time after write_begin()/write_end()
+ * has been called all buffers have been clean and thus they
+ * must have been written at least once. So they are all
+ * mapped and we can happily proceed with mapping them
+ * and writing the page.
+ *
+ * Try to initialize the buffer_heads and check whether
+ * all are mapped and non delay. We don't want to
+ * do block allocation here.
*/
- if (mpd->next_page != mpd->first_page) {
- printk(KERN_INFO
- "flush_alloc_da_page map_blocks: "
- "ino %lu blk %llu, size %u\n",
- mpd->inode->i_ino, mpd->b_blocknr,
- mpd->b_size >> mpd->inode->i_blkbits);
- mpage_da_map_blocks(mpd);
+ ret = block_prepare_write(page, 0, len,
+ noalloc_get_block_write);
+ if (!ret) {
+ page_bufs = page_buffers(page);
+ /* check whether all are mapped and non delay */
+ if (walk_page_buffers(NULL, page_bufs, 0, len, NULL,
+ ext4_bh_delay_or_unwritten)) {
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ return 0;
+ }
+ } else {
/*
- * skip rest of the page in the page_vec
+ * We can't do block allocation here
+ * so just redity the page and unlock
+ * and return
*/
+ redirty_page_for_writepage(wbc, page);
unlock_page(page);
- return MPAGE_DA_EXTENT_TAIL;
+ return 0;
}
+ /* now mark the buffer_heads as dirty and uptodate */
+ block_commit_write(page, 0, len);
+ }

+ if (PageChecked(page) && ext4_should_journal_data(inode)) {
/*
- * Start next extent of pages ...
- */
- mpd->first_page = page->index;
-
- /*
- * ... and blocks
+ * It's mmapped pagecache. Add buffers and journal it. There
+ * doesn't seem much point in redirtying the page here.
*/
- mpd->b_size = 0;
- mpd->b_state = 0;
- mpd->b_blocknr = 0;
+ ClearPageChecked(page);
+ return __ext4_journalled_writepage(page, len);
}

- mpd->next_page = page->index + 1;
- logical = (sector_t) page->index <<
- (PAGE_CACHE_SHIFT - inode->i_blkbits);
+ if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
+ ret = nobh_writepage(page, noalloc_get_block_write, wbc);
+ else if (page_bufs && buffer_uninit(page_bufs)) {
+ ext4_set_bh_endio(page_bufs, inode);
+ ret = block_write_full_page_endio(page, noalloc_get_block_write,
+ wbc, ext4_end_io_buffer_write);
+ } else
+ ret = block_write_full_page(page, noalloc_get_block_write,
+ wbc);

- if (!page_has_buffers(page)) {
- mpage_add_bh_to_extent(mpd, logical, PAGE_CACHE_SIZE,
- (1 << BH_Dirty) | (1 << BH_Uptodate));
- } else {
- /*
- * Page with regular buffer heads, just add all dirty ones
- */
- head = page_buffers(page);
- bh = head;
- do {
- BUG_ON(buffer_locked(bh));
- /*
- * We need to try to allocate
- * unmapped blocks in the same page.
- * Otherwise we won't make progress
- * with the page in ext4_writepage
- */
- if (ext4_bh_delay_or_unwritten(NULL, bh)) {
- mpage_add_bh_to_extent(mpd, logical,
- bh->b_size,
- bh->b_state);
- } else if (buffer_dirty(bh) && (buffer_mapped(bh))) {
- /*
- * mapped dirty buffer. We need to update
- * the b_state because we look at
- * b_state in mpage_da_map_blocks. We don't
- * update b_size because if we find an
- * unmapped buffer_head later we need to
- * use the b_state flag of that buffer_head.
- */
- if (mpd->b_size == 0)
- mpd->b_state = bh->b_state & BH_FLAGS;
- }
- logical++;
- } while ((bh = bh->b_this_page) != head);
- }
- return 0;
+ return ret;
}

-int ext4_alloc_da_blocks(struct inode *inode)
+/*
+ * This is called via ext4_da_writepages() to
+ * calulate the total number of credits to reserve to fit
+ * a single extent allocation into a single transaction,
+ * ext4_da_writpeages() will loop calling this before
+ * the block allocation.
+ */
+
+static int ext4_da_writepages_trans_blocks(struct inode *inode)
{
- struct address_space *mapping = inode->i_mapping;
- struct pagevec pvec;
- pgoff_t index = 0;
+ int max_blocks = EXT4_I(inode)->i_reserved_data_blocks;
+
+ /*
+ * With non-extent format the journal credit needed to
+ * insert nrblocks contiguous block is dependent on
+ * number of contiguous block. So we will limit
+ * number of contiguous block to a sane value
+ */
+ if (!(EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL) &&
+ (max_blocks > EXT4_MAX_TRANS_DATA))
+ max_blocks = EXT4_MAX_TRANS_DATA;
+
+ return ext4_chunk_trans_blocks(inode, max_blocks);
+}
+
+static int ext4_da_writepages(struct address_space *mapping,
+ struct writeback_control *wbc)
+{
+ pgoff_t index;
+ int range_whole = 0;
handle_t *handle = NULL;
struct mpage_da_data mpd;
- int i;
- int nr_pages;
+ struct inode *inode = mapping->host;
+ int no_nrwrite_index_update;
+ int pages_written = 0;
+ long pages_skipped;
+ unsigned int max_pages;
+ int range_cyclic, cycled = 1, io_done = 0;
int needed_blocks, ret = 0;
+ long desired_nr_to_write, nr_to_writebump = 0;
+ loff_t range_start = wbc->range_start;
struct ext4_sb_info *sbi = EXT4_SB(mapping->host->i_sb);

- if (ext4_should_journal_data(inode))
- return 0;
+ trace_ext4_da_writepages(inode, wbc);

/*
- * If no pages to write, return right away.
+ * No pages to write? This is mainly a kludge to avoid starting
+ * a transaction for special inodes like journal inode on last iput()
+ * because that could violate lock ordering on umount
*/
if (!mapping->nrpages || !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
return 0;

/*
- * If the filesystem has aborted, return immediately with an
- * EROFS error.
+ * If the filesystem has aborted, it is read-only, so return
+ * right away instead of dumping stack traces later on that
+ * will obscure the real source of the problem. We test
+ * EXT4_MF_FS_ABORTED instead of sb->s_flag's MS_RDONLY because
+ * the latter could be true if the filesystem is mounted
+ * read-only, and in that case, ext4_da_writepages should
+ * *never* be called, so if that ever happens, we would want
+ * the stack trace.
*/
if (unlikely(sbi->s_mount_flags & EXT4_MF_FS_ABORTED))
return -EROFS;

- printk(KERN_INFO "ext4_alloc_da_pages(%lu)\n", inode->i_ino);
+ if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
+ range_whole = 1;
+
+ range_cyclic = wbc->range_cyclic;
+ if (wbc->range_cyclic) {
+ index = mapping->writeback_index;
+ if (index)
+ cycled = 0;
+ wbc->range_start = index << PAGE_CACHE_SHIFT;
+ wbc->range_end = LLONG_MAX;
+ wbc->range_cyclic = 0;
+ } else
+ index = wbc->range_start >> PAGE_CACHE_SHIFT;
+
+ /*
+ * This works around two forms of stupidity. The first is in
+ * the writeback code, which caps the maximum number of pages
+ * written to be 1024 pages. This is wrong on multiple
+ * levels; different architectues have a different page size,
+ * which changes the maximum amount of data which gets
+ * written. Secondly, 4 megabytes is way too small. XFS
+ * forces this value to be 16 megabytes by multiplying
+ * nr_to_write parameter by four, and then relies on its
+ * allocator to allocate larger extents to make them
+ * contiguous. Unfortunately this brings us to the second
+ * stupidity, which is that ext4's mballoc code only allocates
+ * at most 2048 blocks. So we force contiguous writes up to
+ * the number of dirty blocks in the inode, or
+ * sbi->max_writeback_mb_bump whichever is smaller.
+ */
+ max_pages = sbi->s_max_writeback_mb_bump << (20 - PAGE_CACHE_SHIFT);
+ if (!range_cyclic && range_whole)
+ desired_nr_to_write = wbc->nr_to_write * 8;
+ else
+ desired_nr_to_write = ext4_num_dirty_pages(inode, index,
+ max_pages);
+ if (desired_nr_to_write > max_pages)
+ desired_nr_to_write = max_pages;
+
+ if (wbc->nr_to_write < desired_nr_to_write) {
+ nr_to_writebump = desired_nr_to_write - wbc->nr_to_write;
+ wbc->nr_to_write = desired_nr_to_write;
+ }
+
+ mpd.wbc = wbc;
mpd.inode = mapping->host;

- while (1) {
+ /*
+ * we don't want write_cache_pages to update
+ * nr_to_write and writeback_index
+ */
+ no_nrwrite_index_update = wbc->no_nrwrite_index_update;
+ wbc->no_nrwrite_index_update = 1;
+ pages_skipped = wbc->pages_skipped;
+
+retry:
+ while (!ret && wbc->nr_to_write > 0) {
+
/*
- * we insert one extent at a time. So we need
+ * we insert one extent at a time. So we need
* credit needed for single extent allocation.
* journalled mode is currently not supported
* by delalloc
@@ -3354,67 +3145,48 @@ int ext4_alloc_da_blocks(struct inode *i
BUG_ON(ext4_should_journal_data(inode));
needed_blocks = ext4_da_writepages_trans_blocks(inode);

- pagevec_init(&pvec, 0);
- nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
- PAGECACHE_TAG_DIRTY,
- (pgoff_t)PAGEVEC_SIZE);
- if (nr_pages == 0)
- break;
-
/* start a new transaction*/
handle = ext4_journal_start(inode, needed_blocks);
- if (IS_ERR(handle))
- break;
-
- mpd.b_size = 0;
- mpd.b_state = 0;
- mpd.b_blocknr = 0;
- mpd.first_page = 0;
- mpd.next_page = 0;
- mpd.io_done = 0;
- mpd.pages_written = 0;
- mpd.retval = 0;
-
- do {
- for (i = 0; i < nr_pages; i++) {
- struct page *page = pvec.pages[i];
-
- lock_page(page);
- if (unlikely(page->mapping != mapping) ||
- !PageDirty(page) ||
- PageWriteback(page)) {
- unlock_page(page);
- continue;
- }
-
- ret = flush_alloc_da_page(page, &mpd);
- if (ret) {
- pagevec_release(&pvec);
- goto map_extent;
- }
- }
- pagevec_release(&pvec);
- cond_resched();
-
- nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
- PAGECACHE_TAG_DIRTY,
- (pgoff_t)PAGEVEC_SIZE);
- } while (nr_pages);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ ext4_msg(inode->i_sb, KERN_CRIT, "%s: jbd2_start: "
+ "%ld pages, ino %lu; err %d\n", __func__,
+ wbc->nr_to_write, inode->i_ino, ret);
+ goto out_writepages;
+ }

/*
+ * Now call __mpage_da_writepage to find the next
+ * contiguous region of logical blocks that need
+ * blocks to be allocated by ext4. We don't actually
+ * submit the blocks for I/O here, even though
+ * write_cache_pages thinks it will, and will set the
+ * pages as clean for write before calling
+ * __mpage_da_writepage().
+ */
+ mpd.b_size = 0;
+ mpd.b_state = 0;
+ mpd.b_blocknr = 0;
+ mpd.first_page = 0;
+ mpd.next_page = 0;
+ mpd.io_done = 0;
+ mpd.pages_written = 0;
+ mpd.retval = 0;
+ ret = write_cache_pages(mapping, wbc, __mpage_da_writepage,
+ &mpd);
+ /*
* If we have a contigous extent of pages and we
* haven't done the I/O yet, map the blocks and submit
* them for I/O.
*/
- map_extent:
if (!mpd.io_done && mpd.next_page != mpd.first_page) {
- printk(KERN_INFO
- "ext4_alloc_da_blocks map_blocks: "
- "ino %lu blk %llu, size %u\n",
- mpd.inode->i_ino, mpd.b_blocknr,
- mpd.b_size >> mpd.inode->i_blkbits);
- mpage_da_map_blocks(&mpd);
+ if (mpage_da_map_blocks(&mpd) == 0)
+ mpage_da_submit_io(&mpd);
+ mpd.io_done = 1;
+ ret = MPAGE_DA_EXTENT_TAIL;
}
+ trace_ext4_da_write_pages(inode, &mpd);
+ wbc->nr_to_write -= mpd.pages_written;

ext4_journal_stop(handle);

@@ -3424,484 +3196,707 @@ int ext4_alloc_da_blocks(struct inode *i
* and try again
*/
jbd2_journal_force_commit_nested(sbi->s_journal);
- }
+ wbc->pages_skipped = pages_skipped;
+ ret = 0;
+ } else if (ret == MPAGE_DA_EXTENT_TAIL) {
+ /*
+ * got one extent now try with
+ * rest of the pages
+ */
+ pages_written += mpd.pages_written;
+ wbc->pages_skipped = pages_skipped;
+ ret = 0;
+ io_done = 1;
+ } else if (wbc->nr_to_write)
+ /*
+ * There is no more writeout needed
+ * or we requested for a noblocking writeout
+ * and we found the device congested
+ */
+ break;
}
- printk(KERN_INFO "ext4_alloc_da_pages(%lu) exit\n", inode->i_ino);
+ if (!io_done && !cycled) {
+ cycled = 1;
+ index = 0;
+ wbc->range_start = index << PAGE_CACHE_SHIFT;
+ wbc->range_end = mapping->writeback_index - 1;
+ goto retry;
+ }
+ if (pages_skipped != wbc->pages_skipped)
+ ext4_msg(inode->i_sb, KERN_CRIT,
+ "This should not happen leaving %s "
+ "with nr_to_write = %ld ret = %d\n",
+ __func__, wbc->nr_to_write, ret);
+
+ /* Update index */
+ index += pages_written;
+ wbc->range_cyclic = range_cyclic;
+ if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0))
+ /*
+ * set the writeback_index so that range_cyclic
+ * mode will write it back later
+ */
+ mapping->writeback_index = index;
+
+out_writepages:
+ if (!no_nrwrite_index_update)
+ wbc->no_nrwrite_index_update = 0;
+ if (wbc->nr_to_write > nr_to_writebump)
+ wbc->nr_to_write -= nr_to_writebump;
+ wbc->range_start = range_start;
+ trace_ext4_da_writepages_result(inode, wbc, ret, pages_written);
return ret;
}
-#endif

-/*
- * bmap() is special. It gets used by applications such as lilo and by
- * the swapper to find the on-disk block of a specific piece of data.
- *
- * Naturally, this is dangerous if the block concerned is still in the
- * journal. If somebody makes a swapfile on an ext4 data-journaling
- * filesystem and enables swap, then they may get a nasty shock when the
- * data getting swapped to that swapfile suddenly gets overwritten by
- * the original zero's written out previously to the journal and
- * awaiting writeback in the kernel's buffer cache.
- *
- * So, if we see any bmap calls here on a modified, data-journaled file,
- * take extra steps to flush any blocks which might be in the cache.
- */
-static sector_t ext4_bmap(struct address_space *mapping, sector_t block)
+#define FALL_BACK_TO_NONDELALLOC 1
+static int ext4_nonda_switch(struct super_block *sb)
{
- struct inode *inode = mapping->host;
- journal_t *journal;
- int err;
+ s64 free_blocks, dirty_blocks;
+ struct ext4_sb_info *sbi = EXT4_SB(sb);

- if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY) &&
- test_opt(inode->i_sb, DELALLOC)) {
+ /*
+ * switch to non delalloc mode if we are running low
+ * on free block. The free block accounting via percpu
+ * counters can get slightly wrong with percpu_counter_batch getting
+ * accumulated on each CPU without updating global counters
+ * Delalloc need an accurate free block accounting. So switch
+ * to non delalloc when we are near to error range.
+ */
+ free_blocks = percpu_counter_read_positive(&sbi->s_freeblocks_counter);
+ dirty_blocks =
percpu_counter_read_positive(&sbi->s_dirtyblocks_counter);
+ if (2 * free_blocks < 3 * dirty_blocks ||
+ free_blocks < (dirty_blocks + EXT4_FREEBLOCKS_WATERMARK)) {
/*
- * With delalloc we want to sync the file
- * so that we can make sure we allocate
- * blocks for file
+ * free block count is less that 150% of dirty blocks
+ * or free blocks is less that watermark
*/
- filemap_write_and_wait(mapping);
+ return 1;
}
+ return 0;
+}

- if (EXT4_JOURNAL(inode) && EXT4_I(inode)->i_state & EXT4_STATE_JDATA) {
+static int ext4_da_write_begin(struct file *file, struct
address_space *mapping,
+ loff_t pos, unsigned len, unsigned flags,
+ struct page **pagep, void **fsdata)
+{
+ int ret, retries = 0;
+ struct page *page;
+ pgoff_t index;
+ unsigned from, to;
+ struct inode *inode = mapping->host;
+ handle_t *handle;
+
+ index = pos >> PAGE_CACHE_SHIFT;
+ from = pos & (PAGE_CACHE_SIZE - 1);
+ to = from + len;
+
+ if (ext4_nonda_switch(inode->i_sb)) {
+ *fsdata = (void *)FALL_BACK_TO_NONDELALLOC;
+ return ext4_write_begin(file, mapping, pos,
+ len, flags, pagep, fsdata);
+ }
+ *fsdata = (void *)0;
+ trace_ext4_da_write_begin(inode, pos, len, flags);
+retry:
+ /*
+ * With delayed allocation, we don't log the i_disksize update
+ * if there is delayed block allocation. But we still need
+ * to journalling the i_disksize update if writes to the end
+ * of file which has an already mapped buffer.
+ */
+ handle = ext4_journal_start(inode, 1);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ /* We cannot recurse into the filesystem as the transaction is already
+ * started */
+ flags |= AOP_FLAG_NOFS;
+
+ page = grab_cache_page_write_begin(mapping, index, flags);
+ if (!page) {
+ ext4_journal_stop(handle);
+ ret = -ENOMEM;
+ goto out;
+ }
+ *pagep = page;
+
+ ret = block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
+ ext4_da_get_block_prep);
+ if (ret < 0) {
+ unlock_page(page);
+ ext4_journal_stop(handle);
+ page_cache_release(page);
/*
- * This is a REALLY heavyweight approach, but the use of
- * bmap on dirty files is expected to be extremely rare:
- * only if we run lilo or swapon on a freshly made file
- * do we expect this to happen.
- *
- * (bmap requires CAP_SYS_RAWIO so this does not
- * represent an unprivileged user DOS attack --- we'd be
- * in trouble if mortal users could trigger this path at
- * will.)
- *
- * NB. EXT4_STATE_JDATA is not set on files other than
- * regular files. If somebody wants to bmap a directory
- * or symlink and gets confused because the buffer
- * hasn't yet been flushed to disk, they deserve
- * everything they get.
+ * block_write_begin may have instantiated a few blocks
+ * outside i_size. Trim these off again. Don't need
+ * i_size_read because we hold i_mutex.
*/
+ if (pos + len > inode->i_size)
+ ext4_truncate(inode);
+ }

- EXT4_I(inode)->i_state &= ~EXT4_STATE_JDATA;
- journal = EXT4_JOURNAL(inode);
- jbd2_journal_lock_updates(journal);
- err = jbd2_journal_flush(journal);
- jbd2_journal_unlock_updates(journal);
+ if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
+ goto retry;
+out:
+ return ret;
+}

- if (err)
- return 0;
+/*
+ * Check if we should update i_disksize
+ * when write to the end of file but not require block allocation
+ */
+static int ext4_da_should_update_i_disksize(struct page *page,
+ unsigned long offset)
+{
+ struct buffer_head *bh;
+ struct inode *inode = page->mapping->host;
+ unsigned int idx;
+ int i;
+
+ bh = page_buffers(page);
+ idx = offset >> inode->i_blkbits;
+
+ for (i = 0; i < idx; i++)
+ bh = bh->b_this_page;
+
+ if (!buffer_mapped(bh) || (buffer_delay(bh)) || buffer_unwritten(bh))
+ return 0;
+ return 1;
+}
+
+static int ext4_da_write_end(struct file *file,
+ struct address_space *mapping,
+ loff_t pos, unsigned len, unsigned copied,
+ struct page *page, void *fsdata)
+{
+ struct inode *inode = mapping->host;
+ int ret = 0, ret2;
+ handle_t *handle = ext4_journal_current_handle();
+ loff_t new_i_size;
+ unsigned long start, end;
+ int write_mode = (int)(unsigned long)fsdata;
+
+ if (write_mode == FALL_BACK_TO_NONDELALLOC) {
+ if (ext4_should_order_data(inode)) {
+ return ext4_ordered_write_end(file, mapping, pos,
+ len, copied, page, fsdata);
+ } else if (ext4_should_writeback_data(inode)) {
+ return ext4_writeback_write_end(file, mapping, pos,
+ len, copied, page, fsdata);
+ } else {
+ BUG();
+ }
}

- return generic_block_bmap(mapping, block, ext4_get_block);
-}
+ trace_ext4_da_write_end(inode, pos, len, copied);
+ start = pos & (PAGE_CACHE_SIZE - 1);
+ end = start + copied - 1;

-static int ext4_readpage(struct file *file, struct page *page)
-{
- return mpage_readpage(page, ext4_get_block);
-}
+ /*
+ * generic_write_end() will run mark_inode_dirty() if i_size
+ * changes. So let's piggyback the i_disksize mark_inode_dirty
+ * into that.
+ */

-static int
-ext4_readpages(struct file *file, struct address_space *mapping,
- struct list_head *pages, unsigned nr_pages)
-{
- return mpage_readpages(mapping, pages, nr_pages, ext4_get_block);
+ new_i_size = pos + copied;
+ if (new_i_size > EXT4_I(inode)->i_disksize) {
+ if (ext4_da_should_update_i_disksize(page, end)) {
+ down_write(&EXT4_I(inode)->i_data_sem);
+ if (new_i_size > EXT4_I(inode)->i_disksize) {
+ /*
+ * Updating i_disksize when extending file
+ * without needing block allocation
+ */
+ if (ext4_should_order_data(inode))
+ ret = ext4_jbd2_file_inode(handle,
+ inode);
+
+ EXT4_I(inode)->i_disksize = new_i_size;
+ }
+ up_write(&EXT4_I(inode)->i_data_sem);
+ /* We need to mark inode dirty even if
+ * new_i_size is less that inode->i_size
+ * bu greater than i_disksize.(hint delalloc)
+ */
+ ext4_mark_inode_dirty(handle, inode);
+ }
+ }
+ ret2 = generic_write_end(file, mapping, pos, len, copied,
+ page, fsdata);
+ copied = ret2;
+ if (ret2 < 0)
+ ret = ret2;
+ ret2 = ext4_journal_stop(handle);
+ if (!ret)
+ ret = ret2;
+
+ return ret ? ret : copied;
}

-static void ext4_invalidatepage(struct page *page, unsigned long offset)
+static void ext4_da_invalidatepage(struct page *page, unsigned long offset)
{
- journal_t *journal = EXT4_JOURNAL(page->mapping->host);
-
/*
- * If it's a full truncate we just forget about the pending dirtying
+ * Drop reserved blocks
*/
- if (offset == 0)
- ClearPageChecked(page);
+ BUG_ON(!PageLocked(page));
+ if (!page_has_buffers(page))
+ goto out;

- if (journal)
- jbd2_journal_invalidatepage(journal, page, offset);
- else
- block_invalidatepage(page, offset);
-}
+ ext4_da_page_release_reservation(page, offset);

-static int ext4_releasepage(struct page *page, gfp_t wait)
-{
- journal_t *journal = EXT4_JOURNAL(page->mapping->host);
+out:
+ ext4_invalidatepage(page, offset);

- WARN_ON(PageChecked(page));
- if (!page_has_buffers(page))
- return 0;
- if (journal)
- return jbd2_journal_try_to_free_buffers(journal, page, wait);
- else
- return try_to_free_buffers(page);
+ return;
}

/*
- * O_DIRECT for ext3 (or indirect map) based files
- *
- * If the O_DIRECT write will extend the file then add this inode to the
- * orphan list. So recovery will truncate it back to the original size
- * if the machine crashes during the write.
- *
- * If the O_DIRECT write is intantiating holes inside i_size and the machine
- * crashes then stale disk data _may_ be exposed inside the file. But current
- * VFS code falls back into buffered path in that case so we are safe.
+ * Force all delayed allocation blocks to be allocated for a given inode.
*/
-static ssize_t ext4_ind_direct_IO(int rw, struct kiocb *iocb,
- const struct iovec *iov, loff_t offset,
- unsigned long nr_segs)
+#if 1
+int ext4_alloc_da_blocks(struct inode *inode)
{
- struct file *file = iocb->ki_filp;
- struct inode *inode = file->f_mapping->host;
- struct ext4_inode_info *ei = EXT4_I(inode);
- handle_t *handle;
- ssize_t ret;
- int orphan = 0;
- size_t count = iov_length(iov, nr_segs);
- int retries = 0;
+ trace_ext4_alloc_da_blocks(inode);

- if (rw == WRITE) {
- loff_t final_size = offset + count;
+ if (!EXT4_I(inode)->i_reserved_data_blocks &&
+ !EXT4_I(inode)->i_reserved_meta_blocks)
+ return 0;

- if (final_size > inode->i_size) {
- /* Credits for sb + inode write */
- handle = ext4_journal_start(inode, 2);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
- }
- ret = ext4_orphan_add(handle, inode);
- if (ret) {
- ext4_journal_stop(handle);
- goto out;
- }
- orphan = 1;
- ei->i_disksize = inode->i_size;
- ext4_journal_stop(handle);
+ /*
+ * We do something simple for now. The filemap_flush() will
+ * also start triggering a write of the data blocks, which is
+ * not strictly speaking necessary (and for users of
+ * laptop_mode, not even desirable). However, to do otherwise
+ * would require replicating code paths in:
+ *
+ * ext4_da_writepages() ->
+ * write_cache_pages() ---> (via passed in callback function)
+ * __mpage_da_writepage() -->
+ * mpage_add_bh_to_extent()
+ * mpage_da_map_blocks()
+ *
+ * The problem is that write_cache_pages(), located in
+ * mm/page-writeback.c, marks pages clean in preparation for
+ * doing I/O, which is not desirable if we're not planning on
+ * doing I/O at all.
+ *
+ * We could call write_cache_pages(), and then redirty all of
+ * the pages by calling redirty_page_for_writeback() but that
+ * would be ugly in the extreme. So instead we would need to
+ * replicate parts of the code in the above functions,
+ * simplifying them becuase we wouldn't actually intend to
+ * write out the pages, but rather only collect contiguous
+ * logical block extents, call the multi-block allocator, and
+ * then update the buffer heads with the block allocations.
+ *
+ * For now, though, we'll cheat by calling filemap_flush(),
+ * which will map the blocks, and start the I/O, but not
+ * actually wait for the I/O to complete.
+ */
+ return filemap_flush(inode->i_mapping);
+}
+#else
+static int flush_alloc_da_page(struct page *page, struct mpage_da_data *mpd)
+{
+ struct inode *inode = mpd->inode;
+ struct buffer_head *bh, *head;
+ sector_t logical;
+
+ /*
+ * Can we merge this page to current extent?
+ */
+ if (mpd->next_page != page->index) {
+ /*
+ * Nope, we can't. So, we map non-allocated blocks
+ * and start IO on them using writepage()
+ */
+ if (mpd->next_page != mpd->first_page) {
+ printk(KERN_INFO
+ "flush_alloc_da_page map_blocks: "
+ "ino %lu blk %llu, size %u\n",
+ mpd->inode->i_ino, mpd->b_blocknr,
+ mpd->b_size >> mpd->inode->i_blkbits);
+ mpage_da_map_blocks(mpd);
+ /*
+ * skip rest of the page in the page_vec
+ */
+ unlock_page(page);
+ return MPAGE_DA_EXTENT_TAIL;
}
- }

-retry:
- if (rw == READ && test_opt(inode->i_sb, DIOREAD_NOLOCK)
- && (EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL))
- ret = blockdev_direct_IO_no_locking(rw, iocb, inode,
- inode->i_sb->s_bdev, iov,
- offset, nr_segs,
- ext4_get_block, NULL);
- else
- ret = blockdev_direct_IO(rw, iocb, inode,
- inode->i_sb->s_bdev, iov,
- offset, nr_segs,
- ext4_get_block, NULL);
- if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
- goto retry;
+ /*
+ * Start next extent of pages ...
+ */
+ mpd->first_page = page->index;

- if (orphan) {
- int err;
+ /*
+ * ... and blocks
+ */
+ mpd->b_size = 0;
+ mpd->b_state = 0;
+ mpd->b_blocknr = 0;
+ }

- /* Credits for sb + inode write */
- handle = ext4_journal_start(inode, 2);
- if (IS_ERR(handle)) {
- /* This is really bad luck. We've written the data
- * but cannot extend i_size. Bail out and pretend
- * the write failed... */
- ret = PTR_ERR(handle);
- goto out;
- }
- if (inode->i_nlink)
- ext4_orphan_del(handle, inode);
- if (ret > 0) {
- loff_t end = offset + ret;
- if (end > inode->i_size) {
- ei->i_disksize = end;
- i_size_write(inode, end);
+ mpd->next_page = page->index + 1;
+ logical = (sector_t) page->index <<
+ (PAGE_CACHE_SHIFT - inode->i_blkbits);
+
+ if (!page_has_buffers(page)) {
+ mpage_add_bh_to_extent(mpd, logical, PAGE_CACHE_SIZE,
+ (1 << BH_Dirty) | (1 << BH_Uptodate));
+ } else {
+ /*
+ * Page with regular buffer heads, just add all dirty ones
+ */
+ head = page_buffers(page);
+ bh = head;
+ do {
+ BUG_ON(buffer_locked(bh));
+ /*
+ * We need to try to allocate
+ * unmapped blocks in the same page.
+ * Otherwise we won't make progress
+ * with the page in ext4_writepage
+ */
+ if (ext4_bh_delay_or_unwritten(NULL, bh)) {
+ mpage_add_bh_to_extent(mpd, logical,
+ bh->b_size,
+ bh->b_state);
+ } else if (buffer_dirty(bh) && (buffer_mapped(bh))) {
/*
- * We're going to return a positive `ret'
- * here due to non-zero-length I/O, so there's
- * no way of reporting error returns from
- * ext4_mark_inode_dirty() to userspace. So
- * ignore it.
+ * mapped dirty buffer. We need to update
+ * the b_state because we look at
+ * b_state in mpage_da_map_blocks. We don't
+ * update b_size because if we find an
+ * unmapped buffer_head later we need to
+ * use the b_state flag of that buffer_head.
*/
- ext4_mark_inode_dirty(handle, inode);
+ if (mpd->b_size == 0)
+ mpd->b_state = bh->b_state & BH_FLAGS;
}
- }
- err = ext4_journal_stop(handle);
- if (ret == 0)
- ret = err;
+ logical++;
+ } while ((bh = bh->b_this_page) != head);
}
-out:
- return ret;
+ return 0;
}

-static int ext4_get_block_write(struct inode *inode, sector_t iblock,
- struct buffer_head *bh_result, int create)
+int ext4_alloc_da_blocks(struct inode *inode)
{
- handle_t *handle = ext4_journal_current_handle();
- int ret = 0;
- unsigned max_blocks = bh_result->b_size >> inode->i_blkbits;
- int dio_credits;
- int started = 0;
+ struct address_space *mapping = inode->i_mapping;
+ struct pagevec pvec;
+ pgoff_t index = 0;
+ handle_t *handle = NULL;
+ struct mpage_da_data mpd;
+ int i;
+ int nr_pages;
+ int needed_blocks, ret = 0;
+ struct ext4_sb_info *sbi = EXT4_SB(mapping->host->i_sb);
+
+ if (ext4_should_journal_data(inode))
+ return 0;

- ext4_debug("ext4_get_block_write: inode %lu, create flag %d\n",
- inode->i_ino, create);
/*
- * ext4_get_block in prepare for a DIO write or buffer write.
- * We allocate an uinitialized extent if blocks haven't been allocated.
- * The extent will be converted to initialized after IO complete.
+ * If no pages to write, return right away.
*/
- create = EXT4_GET_BLOCKS_IO_CREATE_EXT;
-
- if (!handle) {
- if (max_blocks > DIO_MAX_BLOCKS)
- max_blocks = DIO_MAX_BLOCKS;
- dio_credits = ext4_chunk_trans_blocks(inode, max_blocks);
- handle = ext4_journal_start(inode, dio_credits);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
- }
- started = 1;
- }
-
- ret = ext4_get_blocks(handle, inode, iblock, max_blocks, bh_result,
- create);
- if (ret > 0) {
- bh_result->b_size = (ret << inode->i_blkbits);
- ret = 0;
- }
- if (started)
- ext4_journal_stop(handle);
-out:
- return ret;
-}
+ if (!mapping->nrpages || !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
+ return 0;

-static void ext4_free_io_end(ext4_io_end_t *io)
-{
- BUG_ON(!io);
- iput(io->inode);
- kfree(io);
-}
+ /*
+ * If the filesystem has aborted, return immediately with an
+ * EROFS error.
+ */
+ if (unlikely(sbi->s_mount_flags & EXT4_MF_FS_ABORTED))
+ return -EROFS;

-static void dump_completed_IO(struct inode * inode)
-{
-#ifdef EXT4_DEBUG
- struct list_head *cur, *before, *after;
- ext4_io_end_t *io, *io0, *io1;
+ printk(KERN_INFO "ext4_alloc_da_pages(%lu)\n", inode->i_ino);
+ mpd.inode = mapping->host;

- if (list_empty(&EXT4_I(inode)->i_completed_io_list)){
- ext4_debug("inode %lu completed_io list is empty\n",
inode->i_ino);
- return;
- }
+ while (1) {
+ /*
+ * we insert one extent at a time. So we need
+ * credit needed for single extent allocation.
+ * journalled mode is currently not supported
+ * by delalloc
+ */
+ BUG_ON(ext4_should_journal_data(inode));
+ needed_blocks = ext4_da_writepages_trans_blocks(inode);

- ext4_debug("Dump inode %lu completed_io list \n", inode->i_ino);
- list_for_each_entry(io, &EXT4_I(inode)->i_completed_io_list, list){
- cur = &io->list;
- before = cur->prev;
- io0 = container_of(before, ext4_io_end_t, list);
- after = cur->next;
- io1 = container_of(after, ext4_io_end_t, list);
+ pagevec_init(&pvec, 0);
+ nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
+ PAGECACHE_TAG_DIRTY,
+ (pgoff_t)PAGEVEC_SIZE);
+ if (nr_pages == 0)
+ break;

- ext4_debug("io 0x%p from inode %lu,prev 0x%p,next 0x%p\n",
- io, inode->i_ino, io0, io1);
- }
-#endif
-}
+ /* start a new transaction*/
+ handle = ext4_journal_start(inode, needed_blocks);
+ if (IS_ERR(handle))
+ break;

-/*
- * check a range of space and convert unwritten extents to written.
- */
-static int ext4_end_io_nolock(ext4_io_end_t *io)
-{
- struct inode *inode = io->inode;
- loff_t offset = io->offset;
- size_t size = io->size;
- int ret = 0;
+ mpd.b_size = 0;
+ mpd.b_state = 0;
+ mpd.b_blocknr = 0;
+ mpd.first_page = 0;
+ mpd.next_page = 0;
+ mpd.io_done = 0;
+ mpd.pages_written = 0;
+ mpd.retval = 0;

- ext4_debug("ext4_end_io_nolock: io 0x%p from inode %lu,list->next 0x%p,"
- "list->prev 0x%p\n",
- io, inode->i_ino, io->list.next, io->list.prev);
+ do {
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];

- if (list_empty(&io->list))
- return ret;
+ lock_page(page);
+ if (unlikely(page->mapping != mapping) ||
+ !PageDirty(page) ||
+ PageWriteback(page)) {
+ unlock_page(page);
+ continue;
+ }

- if (io->flag != EXT4_IO_WRITTEN)
- return ret;
+ ret = flush_alloc_da_page(page, &mpd);
+ if (ret) {
+ pagevec_release(&pvec);
+ goto map_extent;
+ }
+ }
+ pagevec_release(&pvec);
+ cond_resched();

- ret = ext4_convert_unwritten_extents(inode, offset, size);
- if (ret < 0) {
- printk(KERN_EMERG "%s: failed to convert unwritten"
- "extents to written extents, error is %d"
- " io is still on inode %lu aio dio list\n",
- __func__, ret, inode->i_ino);
- return ret;
- }
+ nr_pages = pagevec_lookup_tag(&pvec, mapping, &index,
+ PAGECACHE_TAG_DIRTY,
+ (pgoff_t)PAGEVEC_SIZE);
+ } while (nr_pages);

- /* clear the DIO AIO unwritten flag */
- io->flag = 0;
- return ret;
-}
+ /*
+ * If we have a contigous extent of pages and we
+ * haven't done the I/O yet, map the blocks and submit
+ * them for I/O.
+ */
+ map_extent:
+ if (!mpd.io_done && mpd.next_page != mpd.first_page) {
+ printk(KERN_INFO
+ "ext4_alloc_da_blocks map_blocks: "
+ "ino %lu blk %llu, size %u\n",
+ mpd.inode->i_ino, mpd.b_blocknr,
+ mpd.b_size >> mpd.inode->i_blkbits);
+ mpage_da_map_blocks(&mpd);
+ }

-/*
- * work on completed aio dio IO, to convert unwritten extents to extents
- */
-static void ext4_end_io_work(struct work_struct *work)
-{
- ext4_io_end_t *io = container_of(work, ext4_io_end_t, work);
- struct inode *inode = io->inode;
- int ret = 0;
+ ext4_journal_stop(handle);

- mutex_lock(&inode->i_mutex);
- ret = ext4_end_io_nolock(io);
- if (ret >= 0) {
- if (!list_empty(&io->list))
- list_del_init(&io->list);
- ext4_free_io_end(io);
+ if ((mpd.retval == -ENOSPC) && sbi->s_journal) {
+ /* commit the transaction which would
+ * free blocks released in the transaction
+ * and try again
+ */
+ jbd2_journal_force_commit_nested(sbi->s_journal);
+ }
}
- mutex_unlock(&inode->i_mutex);
+ printk(KERN_INFO "ext4_alloc_da_pages(%lu) exit\n", inode->i_ino);
+ return ret;
}
+#endif

/*
- * This function is called from ext4_sync_file().
+ * bmap() is special. It gets used by applications such as lilo and by
+ * the swapper to find the on-disk block of a specific piece of data.
*
- * When IO is completed, the work to convert unwritten extents to
- * written is queued on workqueue but may not get immediately
- * scheduled. When fsync is called, we need to ensure the
- * conversion is complete before fsync returns.
- * The inode keeps track of a list of pending/completed IO that
- * might needs to do the conversion. This function walks through
- * the list and convert the related unwritten extents for completed IO
- * to written.
- * The function return the number of pending IOs on success.
+ * Naturally, this is dangerous if the block concerned is still in the
+ * journal. If somebody makes a swapfile on an ext4 data-journaling
+ * filesystem and enables swap, then they may get a nasty shock when the
+ * data getting swapped to that swapfile suddenly gets overwritten by
+ * the original zero's written out previously to the journal and
+ * awaiting writeback in the kernel's buffer cache.
+ *
+ * So, if we see any bmap calls here on a modified, data-journaled file,
+ * take extra steps to flush any blocks which might be in the cache.
*/
-int flush_completed_IO(struct inode *inode)
+static sector_t ext4_bmap(struct address_space *mapping, sector_t block)
{
- ext4_io_end_t *io, *tmp;
- int ret = 0;
- int ret2 = 0;
+ struct inode *inode = mapping->host;
+ journal_t *journal;
+ int err;

- if (list_empty(&EXT4_I(inode)->i_completed_io_list))
- return ret;
+ if (mapping_tagged(mapping, PAGECACHE_TAG_DIRTY) &&
+ test_opt(inode->i_sb, DELALLOC)) {
+ /*
+ * With delalloc we want to sync the file
+ * so that we can make sure we allocate
+ * blocks for file
+ */
+ filemap_write_and_wait(mapping);
+ }

- dump_completed_IO(inode);
- list_for_each_entry_safe(io, tmp,
- &EXT4_I(inode)->i_completed_io_list, list) {
- if (io->flag == EXT4_IO_UNWRITTEN)
- continue;
+ if (EXT4_JOURNAL(inode) && EXT4_I(inode)->i_state & EXT4_STATE_JDATA) {
/*
- * Calling ext4_end_io_nolock() to convert completed
- * IO to written.
+ * This is a REALLY heavyweight approach, but the use of
+ * bmap on dirty files is expected to be extremely rare:
+ * only if we run lilo or swapon on a freshly made file
+ * do we expect this to happen.
*
- * When ext4_sync_file() is called, run_queue() may already
- * about to flush the work corresponding to this io structure.
- * It will be upset if it founds the io structure related
- * to the work-to-be schedule is freed.
+ * (bmap requires CAP_SYS_RAWIO so this does not
+ * represent an unprivileged user DOS attack --- we'd be
+ * in trouble if mortal users could trigger this path at
+ * will.)
*
- * Thus we need to keep the io structure still valid here after
- * convertion finished. The io structure has a flag to
- * avoid double converting from both fsync and background work
- * queue work.
+ * NB. EXT4_STATE_JDATA is not set on files other than
+ * regular files. If somebody wants to bmap a directory
+ * or symlink and gets confused because the buffer
+ * hasn't yet been flushed to disk, they deserve
+ * everything they get.
*/
- ret = ext4_end_io_nolock(io);
- if (ret < 0)
- ret2 = ret;
- else
- list_del_init(&io->list);
- }
- return (ret2 < 0) ? ret2 : 0;
-}
-
-static ext4_io_end_t *ext4_init_io_end (struct inode *inode)
-{
- ext4_io_end_t *io = NULL;

- io = kmalloc(sizeof(*io), GFP_NOFS);
+ EXT4_I(inode)->i_state &= ~EXT4_STATE_JDATA;
+ journal = EXT4_JOURNAL(inode);
+ jbd2_journal_lock_updates(journal);
+ err = jbd2_journal_flush(journal);
+ jbd2_journal_unlock_updates(journal);

- if (io) {
- igrab(inode);
- io->inode = inode;
- io->flag = 0;
- io->offset = 0;
- io->size = 0;
- io->error = 0;
- INIT_WORK(&io->work, ext4_end_io_work);
- INIT_LIST_HEAD(&io->list);
+ if (err)
+ return 0;
}

- return io;
+ return generic_block_bmap(mapping, block, ext4_get_block);
}

-static void ext4_end_io_dio(struct kiocb *iocb, loff_t offset,
- ssize_t size, void *private)
+static int ext4_readpage(struct file *file, struct page *page)
{
- ext4_io_end_t *io_end = iocb->private;
- struct workqueue_struct *wq;
-
- /* if not async direct IO or dio with 0 bytes write, just return */
- if (!io_end || !size)
- return;
-
- ext_debug("ext4_end_io_dio(): io_end 0x%p"
- "for inode %lu, iocb 0x%p, offset %llu, size %llu\n",
- iocb->private, io_end->inode->i_ino, iocb, offset,
- size);
+ return mpage_readpage(page, ext4_get_block);
+}

- /* if not aio dio with unwritten extents, just free io and return */
- if (io_end->flag != EXT4_IO_UNWRITTEN){
- ext4_free_io_end(io_end);
- iocb->private = NULL;
- return;
- }
+static int
+ext4_readpages(struct file *file, struct address_space *mapping,
+ struct list_head *pages, unsigned nr_pages)
+{
+ return mpage_readpages(mapping, pages, nr_pages, ext4_get_block);
+}

- io_end->offset = offset;
- io_end->size = size;
- io_end->flag = EXT4_IO_WRITTEN;
- wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;
+static void ext4_invalidatepage(struct page *page, unsigned long offset)
+{
+ journal_t *journal = EXT4_JOURNAL(page->mapping->host);

- /* queue the work to convert unwritten extents to written */
- queue_work(wq, &io_end->work);
+ /*
+ * If it's a full truncate we just forget about the pending dirtying
+ */
+ if (offset == 0)
+ ClearPageChecked(page);

- /* Add the io_end to per-inode completed aio dio list*/
- list_add_tail(&io_end->list,
- &EXT4_I(io_end->inode)->i_completed_io_list);
- iocb->private = NULL;
+ if (journal)
+ jbd2_journal_invalidatepage(journal, page, offset);
+ else
+ block_invalidatepage(page, offset);
}

-static void ext4_end_io_buffer_write(struct buffer_head *bh, int uptodate)
+static int ext4_releasepage(struct page *page, gfp_t wait)
{
- ext4_io_end_t *io_end = bh->b_private;
- struct workqueue_struct *wq;
+ journal_t *journal = EXT4_JOURNAL(page->mapping->host);

- if (!io_end)
- goto out;
- io_end->flag = EXT4_IO_WRITTEN;
- wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;
- /* queue the work to convert unwritten extents to written */
- queue_work(wq, &io_end->work);
-out:
- bh->b_private = NULL;
- bh->b_end_io = NULL;
- clear_buffer_uninit(bh);
- end_buffer_async_write(bh, uptodate);
+ WARN_ON(PageChecked(page));
+ if (!page_has_buffers(page))
+ return 0;
+ if (journal)
+ return jbd2_journal_try_to_free_buffers(journal, page, wait);
+ else
+ return try_to_free_buffers(page);
}

-static int ext4_set_bh_endio(struct buffer_head *bh, struct inode *inode)
+/*
+ * O_DIRECT for ext3 (or indirect map) based files
+ *
+ * If the O_DIRECT write will extend the file then add this inode to the
+ * orphan list. So recovery will truncate it back to the original size
+ * if the machine crashes during the write.
+ *
+ * If the O_DIRECT write is intantiating holes inside i_size and the machine
+ * crashes then stale disk data _may_ be exposed inside the file. But current
+ * VFS code falls back into buffered path in that case so we are safe.
+ */
+static ssize_t ext4_ind_direct_IO(int rw, struct kiocb *iocb,
+ const struct iovec *iov, loff_t offset,
+ unsigned long nr_segs)
{
- ext4_io_end_t *io_end;
- struct page *page = bh->b_page;
- loff_t offset = (sector_t)page->index << PAGE_CACHE_SHIFT;
- size_t size = bh->b_size;
+ struct file *file = iocb->ki_filp;
+ struct inode *inode = file->f_mapping->host;
+ struct ext4_inode_info *ei = EXT4_I(inode);
+ handle_t *handle;
+ ssize_t ret;
+ int orphan = 0;
+ size_t count = iov_length(iov, nr_segs);
+ int retries = 0;

- io_end = ext4_init_io_end(inode);
- if (!io_end)
- return -ENOMEM;
- io_end->offset = offset;
- io_end->size = size;
- io_end->flag = EXT4_IO_UNWRITTEN;
- /* Add the io_end to per-inode completed io list*/
- list_add_tail(&io_end->list,
- &EXT4_I(io_end->inode)->i_completed_io_list);
+ if (rw == WRITE) {
+ loff_t final_size = offset + count;

- bh->b_private = io_end;
- bh->b_end_io = ext4_end_io_buffer_write;
- return 0;
+ if (final_size > inode->i_size) {
+ /* Credits for sb + inode write */
+ handle = ext4_journal_start(inode, 2);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ ret = ext4_orphan_add(handle, inode);
+ if (ret) {
+ ext4_journal_stop(handle);
+ goto out;
+ }
+ orphan = 1;
+ ei->i_disksize = inode->i_size;
+ ext4_journal_stop(handle);
+ }
+ }
+
+retry:
+ if (rw == READ && test_opt(inode->i_sb, DIOREAD_NOLOCK)
+ && (EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL))
+ ret = blockdev_direct_IO_no_locking(rw, iocb, inode,
+ inode->i_sb->s_bdev, iov,
+ offset, nr_segs,
+ ext4_get_block, NULL);
+ else
+ ret = blockdev_direct_IO(rw, iocb, inode,
+ inode->i_sb->s_bdev, iov,
+ offset, nr_segs,
+ ext4_get_block, NULL);
+ if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
+ goto retry;
+
+ if (orphan) {
+ int err;
+
+ /* Credits for sb + inode write */
+ handle = ext4_journal_start(inode, 2);
+ if (IS_ERR(handle)) {
+ /* This is really bad luck. We've written the data
+ * but cannot extend i_size. Bail out and pretend
+ * the write failed... */
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ if (inode->i_nlink)
+ ext4_orphan_del(handle, inode);
+ if (ret > 0) {
+ loff_t end = offset + ret;
+ if (end > inode->i_size) {
+ ei->i_disksize = end;
+ i_size_write(inode, end);
+ /*
+ * We're going to return a positive `ret'
+ * here due to non-zero-length I/O, so there's
+ * no way of reporting error returns from
+ * ext4_mark_inode_dirty() to userspace. So
+ * ignore it.
+ */
+ ext4_mark_inode_dirty(handle, inode);
+ }
+ }
+ err = ext4_journal_stop(handle);
+ if (ret == 0)
+ ret = err;
+ }
+out:
+ return ret;
}

/*
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[RFC PATCH 3/4] ext4: use direct_IO_no_locking in ext4 DIO read

2009-12-15T17:40:54Z

ext4: use direct_IO_no_locking in ext4 dio read.

Signed-off-by: Jiaying Zhang <jiayingz@...>
---
fs/ext4/inode.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

Index: git-ext4/fs/ext4/inode.c
===================================================================
--- git-ext4.orig/fs/ext4/inode.c 2009-12-15 16:03:15.000000000 -0800
+++ git-ext4/fs/ext4/inode.c 2009-12-15 16:04:03.000000000 -0800
@@ -3580,7 +3580,15 @@ static ssize_t ext4_ind_direct_IO(int rw
}

retry:
- ret = blockdev_direct_IO(rw, iocb, inode, inode->i_sb->s_bdev, iov,
+ if (rw == READ && test_opt(inode->i_sb, DIOREAD_NOLOCK)
+ && (EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL))
+ ret = blockdev_direct_IO_no_locking(rw, iocb, inode,
+ inode->i_sb->s_bdev, iov,
+ offset, nr_segs,
+ ext4_get_block, NULL);
+ else
+ ret = blockdev_direct_IO(rw, iocb, inode,
+ inode->i_sb->s_bdev, iov,
offset, nr_segs,
ext4_get_block, NULL);
if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[RFC PATCH 2/4] ext4: use ext4_get_block_write in buffer write

2009-12-15T17:39:08Z

ext4: use ext4_get_block_write in buffer write

Allocate uninitialized extent before ext4 buffer write and
convert the extent to initialized after io completes.
The purpose is to make sure an extent can only be marked
initialized after it has been written with new data so
we can safely drop the i_mutex lock in ext4 DIO read without
exposing stale data. This helps to improve multi-thread DIO
read performance on high-speed disks.

Skip the nobh and data=journal mount cases to make things simple for now.

Signed-off-by: Jiaying Zhang <jiayingz@...>
---
fs/ext4/ext4.h | 5 +++
fs/ext4/extents.c | 10 +++---
fs/ext4/fsync.c | 3 ++
fs/ext4/inode.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++--------
fs/ext4/super.c | 30 +++++++++++++++++---
5 files changed, 108 insertions(+), 21 deletions(-)

Index: git-ext4/fs/ext4/extents.c
===================================================================
--- git-ext4.orig/fs/ext4/extents.c 2009-12-15 16:03:05.000000000 -0800
+++ git-ext4/fs/ext4/extents.c 2009-12-15 16:03:15.000000000 -0800
@@ -3052,6 +3052,7 @@ ext4_ext_handle_uninitialized_extents(ha
io->flag = EXT4_IO_UNWRITTEN;
else
EXT4_I(inode)->i_state |= EXT4_STATE_DIO_UNWRITTEN;
+ set_buffer_uninit(bh_result);
goto out;
}
/* IO end_io complete, convert the filled extent to written */
@@ -3291,11 +3292,9 @@ int ext4_ext_get_blocks(handle_t *handle
if (flags & EXT4_GET_BLOCKS_UNINIT_EXT){
ext4_ext_mark_uninitialized(&newex);
/*
- * io_end structure was created for every async
- * direct IO write to the middle of the file.
- * To avoid unecessary convertion for every aio dio rewrite
- * to the mid of file, here we flag the IO that is really
- * need the convertion.
+ * io_end structure was created for every IO write to an
+ * uninitialized extent. To avoid unecessary convertion,
+ * here we flag the IO that really needs the convertion.
* For non asycn direct IO case, flag the inode state
* that we need to perform convertion when IO is done.
*/
@@ -3306,6 +3305,7 @@ int ext4_ext_get_blocks(handle_t *handle
EXT4_I(inode)->i_state |=
EXT4_STATE_DIO_UNWRITTEN;;
}
+ set_buffer_uninit(bh_result);
}
err = ext4_ext_insert_extent(handle, inode, path, &newex, flags);
if (err) {
Index: git-ext4/fs/ext4/ext4.h
===================================================================
--- git-ext4.orig/fs/ext4/ext4.h 2009-12-15 16:03:05.000000000 -0800
+++ git-ext4/fs/ext4/ext4.h 2009-12-15 16:03:15.000000000 -0800
@@ -134,6 +134,7 @@ struct mpage_da_data {
int retval;
};
#define EXT4_IO_UNWRITTEN 0x1
+#define EXT4_IO_WRITTEN 0x2
typedef struct ext4_io_end {
struct list_head list; /* per-file finished AIO list */
struct inode *inode; /* file being written to */
@@ -752,6 +753,7 @@ struct ext4_inode_info {
#define EXT4_MOUNT_QUOTA 0x80000 /* Some quota option set */
#define EXT4_MOUNT_USRQUOTA 0x100000 /* "old" user quota */
#define EXT4_MOUNT_GRPQUOTA 0x200000 /* "old" group quota */
+#define EXT4_MOUNT_DIOREAD_NOLOCK 0x400000 /* Enable support for
dio read nolocking */
#define EXT4_MOUNT_JOURNAL_CHECKSUM 0x800000 /* Journal checksums */
#define EXT4_MOUNT_JOURNAL_ASYNC_COMMIT 0x1000000 /* Journal
Async Commit */
#define EXT4_MOUNT_I_VERSION 0x2000000 /* i_version support */
@@ -1764,6 +1766,9 @@ static inline void set_bitmap_uptodate(s
set_bit(BH_BITMAP_UPTODATE, &(bh)->b_state);
}

+/* BH_Uninit flag: blocks are allocated but uninitialized on disk */
+#define BH_Uninit (BH_JBDPrivateStart + 1)
+BUFFER_FNS(Uninit, uninit)
#endif /* __KERNEL__ */

#endif /* _EXT4_H */
Index: git-ext4/fs/ext4/super.c
===================================================================
--- git-ext4.orig/fs/ext4/super.c 2009-12-15 16:03:05.000000000 -0800
+++ git-ext4/fs/ext4/super.c 2009-12-15 16:03:15.000000000 -0800
@@ -921,6 +921,9 @@ static int ext4_show_options(struct seq_
if (test_opt(sb, NOLOAD))
seq_puts(seq, ",norecovery");

+ if (test_opt(sb, DIOREAD_NOLOCK))
+ seq_puts(seq, ",dioread_nolock");
+
ext4_show_quota_options(seq, sb);

return 0;
@@ -1103,6 +1106,7 @@ enum {
Opt_stripe, Opt_delalloc, Opt_nodelalloc,
Opt_block_validity, Opt_noblock_validity,
Opt_inode_readahead_blks, Opt_journal_ioprio,
+ Opt_dioread_nolock, Opt_dioread_lock,
Opt_discard, Opt_nodiscard, Opt_akpm_lock_hack
};

@@ -1171,6 +1175,8 @@ static const match_table_t tokens = {
{Opt_auto_da_alloc, "auto_da_alloc=%u"},
{Opt_auto_da_alloc, "auto_da_alloc"},
{Opt_noauto_da_alloc, "noauto_da_alloc"},
+ {Opt_dioread_nolock, "dioread_nolock"},
+ {Opt_dioread_lock, "dioread_lock"},
{Opt_discard, "discard"},
{Opt_nodiscard, "nodiscard"},
{Opt_err, NULL},
@@ -1603,6 +1609,12 @@ set_qf_format:
case Opt_nodiscard:
clear_opt(sbi->s_mount_opt, DISCARD);
break;
+ case Opt_dioread_nolock:
+ set_opt(sbi->s_mount_opt, DIOREAD_NOLOCK);
+ break;
+ case Opt_dioread_lock:
+ clear_opt(sbi->s_mount_opt, DIOREAD_NOLOCK);
+ break;
case Opt_akpm_lock_hack:
set_opt(sbi->s_mount_opt, AKPM_LOCK_HACK);
break;
@@ -2769,7 +2781,7 @@ static int ext4_fill_super(struct super_
EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER)) {
ext4_msg(sb, KERN_ERR, "required journal recovery "
"suppressed and not mounted read-only");
- goto failed_mount4;
+ goto failed_mount_wq;
} else {
clear_opt(sbi->s_mount_opt, DATA_FLAGS);
set_opt(sbi->s_mount_opt, WRITEBACK_DATA);
@@ -2782,7 +2794,7 @@ static int ext4_fill_super(struct super_
!jbd2_journal_set_features(EXT4_SB(sb)->s_journal, 0, 0,
JBD2_FEATURE_INCOMPAT_64BIT)) {
ext4_msg(sb, KERN_ERR, "Failed to set 64-bit journal feature");
- goto failed_mount4;
+ goto failed_mount_wq;
}

if (test_opt(sb, JOURNAL_ASYNC_COMMIT)) {
@@ -2821,7 +2833,7 @@ static int ext4_fill_super(struct super_
(sbi->s_journal, 0, 0, JBD2_FEATURE_INCOMPAT_REVOKE)) {
ext4_msg(sb, KERN_ERR, "Journal does not support "
"requested data journaling mode");
- goto failed_mount4;
+ goto failed_mount_wq;
}
default:
break;
@@ -2829,13 +2841,17 @@ static int ext4_fill_super(struct super_
set_task_ioprio(sbi->s_journal->j_task, journal_ioprio);

no_journal:
-
if (test_opt(sb, NOBH)) {
if (!(test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_WRITEBACK_DATA)) {
ext4_msg(sb, KERN_WARNING, "Ignoring nobh option - "
"its supported only with writeback mode");
clear_opt(sbi->s_mount_opt, NOBH);
}
+ if (test_opt(sb, DIOREAD_NOLOCK)) {
+ ext4_msg(sb, KERN_WARNING, "dioread_nolock option is "
+ "not supported with nobh mode");
+ goto failed_mount_wq;
+ }
}
EXT4_SB(sb)->dio_unwritten_wq = create_workqueue("ext4-dio-unwritten");
if (!EXT4_SB(sb)->dio_unwritten_wq) {
@@ -2900,6 +2916,12 @@ no_journal:
"requested data journaling mode");
clear_opt(sbi->s_mount_opt, DELALLOC);
}
+ if (test_opt(sb, DIOREAD_NOLOCK) &&
+ (test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA)) {
+ ext4_msg(sb, KERN_WARNING, "Ignoring dioread_nolock option - "
+ "requested data journaling mode");
+ clear_opt(sbi->s_mount_opt, DIOREAD_NOLOCK);
+ }

err = ext4_setup_system_zone(sb);
if (err) {
Index: git-ext4/fs/ext4/inode.c
===================================================================
--- git-ext4.orig/fs/ext4/inode.c 2009-12-15 16:03:05.000000000 -0800
+++ git-ext4/fs/ext4/inode.c 2009-12-15 16:03:15.000000000 -0800
@@ -1492,6 +1492,8 @@ static int do_journal_get_write_access(h
return ext4_journal_get_write_access(handle, bh);
}

+static int ext4_get_block_write(struct inode *inode, sector_t iblock,
+ struct buffer_head *bh_result, int create);
static int ext4_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata)
@@ -1533,8 +1535,12 @@ retry:
}
*pagep = page;

- ret = block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
- ext4_get_block);
+ if (test_opt(inode->i_sb, DIOREAD_NOLOCK))
+ ret = block_write_begin(file, mapping, pos, len, flags, pagep,
+ fsdata, ext4_get_block_write);
+ else
+ ret = block_write_begin(file, mapping, pos, len, flags, pagep,
+ fsdata, ext4_get_block);

if (!ret && ext4_should_journal_data(inode)) {
ret = walk_page_buffers(handle, page_buffers(page),
@@ -2053,6 +2059,8 @@ static void mpage_put_bnr_to_bhs(struct
} else if (buffer_mapped(bh))
BUG_ON(bh->b_blocknr != pblock);

+ if (buffer_uninit(exbh))
+ set_buffer_uninit(bh);
cur_logical++;
pblock++;
} while ((bh = bh->b_this_page) != head);
@@ -2183,6 +2191,8 @@ static int mpage_da_map_blocks(struct mp
new.b_state = 0;
get_blocks_flags = (EXT4_GET_BLOCKS_CREATE |
EXT4_GET_BLOCKS_DELALLOC_RESERVE);
+ if (test_opt(mpd->inode->i_sb, DIOREAD_NOLOCK))
+ get_blocks_flags |= EXT4_GET_BLOCKS_IO_CREATE_EXT;
if (mpd->b_state & (1 << BH_Delay))
get_blocks_flags |= EXT4_GET_BLOCKS_UPDATE_RESERVE_SPACE;
blks = ext4_get_blocks(handle, mpd->inode, next, max_blocks,
@@ -2597,6 +2607,9 @@ out:
return ret;
}

+static int ext4_set_bh_endio(struct buffer_head *bh, struct inode *inode);
+static void ext4_end_io_buffer_write(struct buffer_head *bh, int uptodate);
+
/*
* Note that we don't need to start a transaction unless we're journaling data
* because we should have holes filled from ext4_page_mkwrite(). We even don't
@@ -2644,7 +2657,7 @@ static int ext4_writepage(struct page *p
int ret = 0;
loff_t size;
unsigned int len;
- struct buffer_head *page_bufs;
+ struct buffer_head *page_bufs = NULL;
struct inode *inode = page->mapping->host;

trace_ext4_writepage(inode, page);
@@ -2720,7 +2733,11 @@ static int ext4_writepage(struct page *p

if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
ret = nobh_writepage(page, noalloc_get_block_write, wbc);
- else
+ else if (page_bufs && buffer_uninit(page_bufs)) {
+ ext4_set_bh_endio(page_bufs, inode);
+ ret = block_write_full_page_endio(page, noalloc_get_block_write,
+ wbc, ext4_end_io_buffer_write);
+ } else
ret = block_write_full_page(page, noalloc_get_block_write,
wbc);

@@ -3697,12 +3714,10 @@ static int ext4_end_io_nolock(ext4_io_en
if (list_empty(&io->list))
return ret;

- if (io->flag != EXT4_IO_UNWRITTEN)
+ if (io->flag != EXT4_IO_WRITTEN)
return ret;

- if (offset + size <= i_size_read(inode))
- ret = ext4_convert_unwritten_extents(inode, offset, size);
-
+ ret = ext4_convert_unwritten_extents(inode, offset, size);
if (ret < 0) {
printk(KERN_EMERG "%s: failed to convert unwritten"
"extents to written extents, error is %d"
@@ -3750,7 +3765,7 @@ static void ext4_end_io_work(struct work
*/
int flush_completed_IO(struct inode *inode)
{
- ext4_io_end_t *io;
+ ext4_io_end_t *io, *tmp;
int ret = 0;
int ret2 = 0;

@@ -3758,9 +3773,10 @@ int flush_completed_IO(struct inode *ino
return ret;

dump_completed_IO(inode);
- while (!list_empty(&EXT4_I(inode)->i_completed_io_list)){
- io = list_entry(EXT4_I(inode)->i_completed_io_list.next,
- ext4_io_end_t, list);
+ list_for_each_entry_safe(io, tmp,
+ &EXT4_I(inode)->i_completed_io_list, list) {
+ if (io->flag == EXT4_IO_UNWRITTEN)
+ continue;
/*
* Calling ext4_end_io_nolock() to convert completed
* IO to written.
@@ -3828,6 +3844,7 @@ static void ext4_end_io_dio(struct kiocb

io_end->offset = offset;
io_end->size = size;
+ io_end->flag = EXT4_IO_WRITTEN;
wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;

/* queue the work to convert unwritten extents to written */
@@ -3839,6 +3856,46 @@ static void ext4_end_io_dio(struct kiocb
iocb->private = NULL;
}

+static void ext4_end_io_buffer_write(struct buffer_head *bh, int uptodate)
+{
+ ext4_io_end_t *io_end = bh->b_private;
+ struct workqueue_struct *wq;
+
+ if (!io_end)
+ goto out;
+ io_end->flag = EXT4_IO_WRITTEN;
+ wq = EXT4_SB(io_end->inode->i_sb)->dio_unwritten_wq;
+ /* queue the work to convert unwritten extents to written */
+ queue_work(wq, &io_end->work);
+out:
+ bh->b_private = NULL;
+ bh->b_end_io = NULL;
+ clear_buffer_uninit(bh);
+ end_buffer_async_write(bh, uptodate);
+}
+
+static int ext4_set_bh_endio(struct buffer_head *bh, struct inode *inode)
+{
+ ext4_io_end_t *io_end;
+ struct page *page = bh->b_page;
+ loff_t offset = (sector_t)page->index << PAGE_CACHE_SHIFT;
+ size_t size = bh->b_size;
+
+ io_end = ext4_init_io_end(inode);
+ if (!io_end)
+ return -ENOMEM;
+ io_end->offset = offset;
+ io_end->size = size;
+ io_end->flag = EXT4_IO_UNWRITTEN;
+ /* Add the io_end to per-inode completed io list*/
+ list_add_tail(&io_end->list,
+ &EXT4_I(io_end->inode)->i_completed_io_list);
+
+ bh->b_private = io_end;
+ bh->b_end_io = ext4_end_io_buffer_write;
+ return 0;
+}
+
/*
* For ext4 extent files, ext4 will do direct-io write to holes,
* preallocated extents, and those write extend the file, no need to
Index: git-ext4/fs/ext4/fsync.c
===================================================================
--- git-ext4.orig/fs/ext4/fsync.c 2009-12-15 16:03:05.000000000 -0800
+++ git-ext4/fs/ext4/fsync.c 2009-12-15 16:03:15.000000000 -0800
@@ -101,6 +101,9 @@ int ext4_sync_file(struct file *file, st
if (ret == 0)
ret = err;
}
+ if (ret == 0)
+ flush_completed_IO(inode);
+ BUG_ON(!list_empty(&EXT4_I(inode)->i_completed_io_list));
out:
if (journal && (journal->j_flags & JBD2_BARRIER))
blkdev_issue_flush(inode->i_sb->s_bdev, NULL);
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[RFC PATCH 1/4] ext4: DIO get_block code cleanup

2009-12-15T17:37:53Z

ext4: dio get_block code cleanup in prepare for it to be used by buffer write

Renaming the dio block allocation flags, variables and functions
introduced in Mingming's "Direct IO for holes and fallocate"
patches so that they can be used by ext4 buffer write as well.

Signed-off-by: Jiaying Zhang <jiayingz@...>
---
fs/ext4/ext4.h | 18 ++++----
fs/ext4/extents.c | 24 +++++------
fs/ext4/fsync.c | 2
fs/ext4/inode.c | 112 ++++++++++++++++++++++++++----------------------------
fs/ext4/super.c | 2
5 files changed, 78 insertions(+), 80 deletions(-)

Index: git-ext4/fs/ext4/extents.c
===================================================================
--- git-ext4.orig/fs/ext4/extents.c 2009-12-15 15:14:34.000000000 -0800
+++ git-ext4/fs/ext4/extents.c 2009-12-15 16:03:05.000000000 -0800
@@ -1603,7 +1603,7 @@ int ext4_ext_insert_extent(handle_t *han
BUG_ON(path[depth].p_hdr == NULL);

/* try to insert block into found extent and return */
- if (ex && (flag != EXT4_GET_BLOCKS_DIO_CREATE_EXT)
+ if (ex && !(flag & EXT4_GET_BLOCKS_PRE_IO)
&& ext4_can_extents_be_merged(inode, ex, newext)) {
ext_debug("append [%d]%d block to %d:[%d]%d (from %llu)\n",
ext4_ext_is_uninitialized(newext),
@@ -1724,7 +1724,7 @@ has_space:

merge:
/* try to merge extents to the right */
- if (flag != EXT4_GET_BLOCKS_DIO_CREATE_EXT)
+ if (!(flag & EXT4_GET_BLOCKS_PRE_IO))
ext4_ext_try_to_merge(inode, path, nearex);

/* try to merge extents to the left */
@@ -2966,7 +2966,7 @@ fix_extent_len:
ext4_ext_dirty(handle, inode, path + depth);
return err;
}
-static int ext4_convert_unwritten_extents_dio(handle_t *handle,
+static int ext4_convert_unwritten_extents_endio(handle_t *handle,
struct inode *inode,
struct ext4_ext_path *path)
{
@@ -3038,8 +3038,8 @@ ext4_ext_handle_uninitialized_extents(ha
flags, allocated);
ext4_ext_show_leaf(inode, path);

- /* DIO get_block() before submit the IO, split the extent */
- if (flags == EXT4_GET_BLOCKS_DIO_CREATE_EXT) {
+ /* get_block() before submit the IO, split the extent */
+ if ((flags & EXT4_GET_BLOCKS_PRE_IO)) {
ret = ext4_split_unwritten_extents(handle,
inode, path, iblock,
max_blocks, flags);
@@ -3049,14 +3049,14 @@ ext4_ext_handle_uninitialized_extents(ha
* completed
*/
if (io)
- io->flag = DIO_AIO_UNWRITTEN;
+ io->flag = EXT4_IO_UNWRITTEN;
else
EXT4_I(inode)->i_state |= EXT4_STATE_DIO_UNWRITTEN;
goto out;
}
- /* async DIO end_io complete, convert the filled extent to written */
- if (flags == EXT4_GET_BLOCKS_DIO_CONVERT_EXT) {
- ret = ext4_convert_unwritten_extents_dio(handle, inode,
+ /* IO end_io complete, convert the filled extent to written */
+ if ((flags & EXT4_GET_BLOCKS_CONVERT)) {
+ ret = ext4_convert_unwritten_extents_endio(handle, inode,
path);
goto out2;
}
@@ -3299,9 +3299,9 @@ int ext4_ext_get_blocks(handle_t *handle
* For non asycn direct IO case, flag the inode state
* that we need to perform convertion when IO is done.
*/
- if (flags == EXT4_GET_BLOCKS_DIO_CREATE_EXT) {
+ if ((flags & EXT4_GET_BLOCKS_PRE_IO)) {
if (io)
- io->flag = DIO_AIO_UNWRITTEN;
+ io->flag = EXT4_IO_UNWRITTEN;
else
EXT4_I(inode)->i_state |=
EXT4_STATE_DIO_UNWRITTEN;;
@@ -3561,7 +3561,7 @@ int ext4_convert_unwritten_extents(struc
map_bh.b_state = 0;
ret = ext4_get_blocks(handle, inode, block,
max_blocks, &map_bh,
- EXT4_GET_BLOCKS_DIO_CONVERT_EXT);
+ EXT4_GET_BLOCKS_IO_CONVERT_EXT);
if (ret <= 0) {
WARN_ON(ret <= 0);
printk(KERN_ERR "%s: ext4_ext_get_blocks "
Index: git-ext4/fs/ext4/ext4.h
===================================================================
--- git-ext4.orig/fs/ext4/ext4.h 2009-12-15 15:14:34.000000000 -0800
+++ git-ext4/fs/ext4/ext4.h 2009-12-15 16:03:05.000000000 -0800
@@ -133,7 +133,7 @@ struct mpage_da_data {
int pages_written;
int retval;
};
-#define DIO_AIO_UNWRITTEN 0x1
+#define EXT4_IO_UNWRITTEN 0x1
typedef struct ext4_io_end {
struct list_head list; /* per-file finished AIO list */
struct inode *inode; /* file being written to */
@@ -367,13 +367,13 @@ struct ext4_new_group_data {
/* caller is from the direct IO path, request to creation of an
unitialized extents if not allocated, split the uninitialized
extent if blocks has been preallocated already*/
-#define EXT4_GET_BLOCKS_DIO 0x0010
+#define EXT4_GET_BLOCKS_PRE_IO 0x0010
#define EXT4_GET_BLOCKS_CONVERT 0x0020
-#define EXT4_GET_BLOCKS_DIO_CREATE_EXT (EXT4_GET_BLOCKS_DIO|\
+#define EXT4_GET_BLOCKS_IO_CREATE_EXT (EXT4_GET_BLOCKS_PRE_IO|\
+ EXT4_GET_BLOCKS_CREATE_UNINIT_EXT)
+ /* Convert extent to initialized after IO complete */
+#define EXT4_GET_BLOCKS_IO_CONVERT_EXT (EXT4_GET_BLOCKS_CONVERT|\
EXT4_GET_BLOCKS_CREATE_UNINIT_EXT)
- /* Convert extent to initialized after direct IO complete */
-#define EXT4_GET_BLOCKS_DIO_CONVERT_EXT
(EXT4_GET_BLOCKS_CONVERT|\
- EXT4_GET_BLOCKS_DIO_CREATE_EXT)

/*
* Flags used by ext4_free_blocks
@@ -707,8 +707,8 @@ struct ext4_inode_info {

spinlock_t i_block_reservation_lock;

- /* completed async DIOs that might need unwritten extents handling */
- struct list_head i_aio_dio_complete_list;
+ /* completed IOs that might need unwritten extents handling */
+ struct list_head i_completed_io_list;
/* current io_end structure for async DIO write*/
ext4_io_end_t *cur_aio_dio;
};
@@ -1432,7 +1432,7 @@ extern int ext4_block_truncate_page(hand
struct address_space *mapping, loff_t from);
extern int ext4_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf);
extern qsize_t ext4_get_reserved_space(struct inode *inode);
-extern int flush_aio_dio_completed_IO(struct inode *inode);
+extern int flush_completed_IO(struct inode *inode);
/* ioctl.c */
extern long ext4_ioctl(struct file *, unsigned int, unsigned long);
extern long ext4_compat_ioctl(struct file *, unsigned int, unsigned long);
Index: git-ext4/fs/ext4/super.c
===================================================================
--- git-ext4.orig/fs/ext4/super.c 2009-12-15 15:14:34.000000000 -0800
+++ git-ext4/fs/ext4/super.c 2009-12-15 16:03:05.000000000 -0800
@@ -711,7 +711,7 @@ static struct inode *ext4_alloc_inode(st
ei->i_allocated_meta_blocks = 0;
ei->i_delalloc_reserved_flag = 0;
spin_lock_init(&(ei->i_block_reservation_lock));
- INIT_LIST_HEAD(&ei->i_aio_dio_complete_list);
+ INIT_LIST_HEAD(&ei->i_completed_io_list);
ei->cur_aio_dio = NULL;

return &ei->vfs_inode;
Index: git-ext4/fs/ext4/fsync.c
===================================================================
--- git-ext4.orig/fs/ext4/fsync.c 2009-12-15 15:14:34.000000000 -0800
+++ git-ext4/fs/ext4/fsync.c 2009-12-15 16:03:05.000000000 -0800
@@ -58,7 +58,7 @@ int ext4_sync_file(struct file *file, st

trace_ext4_sync_file(file, dentry, datasync);

- ret = flush_aio_dio_completed_IO(inode);
+ ret = flush_completed_IO(inode);
if (ret < 0)
return ret;
/*
Index: git-ext4/fs/ext4/inode.c
===================================================================
--- git-ext4.orig/fs/ext4/inode.c 2009-12-15 15:14:34.000000000 -0800
+++ git-ext4/fs/ext4/inode.c 2009-12-15 16:03:05.000000000 -0800
@@ -3606,52 +3606,44 @@ out:
return ret;
}

-static int ext4_get_block_dio_write(struct inode *inode, sector_t iblock,
+static int ext4_get_block_write(struct inode *inode, sector_t iblock,
struct buffer_head *bh_result, int create)
{
- handle_t *handle = NULL;
+ handle_t *handle = ext4_journal_current_handle();
int ret = 0;
unsigned max_blocks = bh_result->b_size >> inode->i_blkbits;
int dio_credits;
+ int started = 0;

- ext4_debug("ext4_get_block_dio_write: inode %lu, create flag %d\n",
+ ext4_debug("ext4_get_block_write: inode %lu, create flag %d\n",
inode->i_ino, create);
/*
- * DIO VFS code passes create = 0 flag for write to
- * the middle of file. It does this to avoid block
- * allocation for holes, to prevent expose stale data
- * out when there is parallel buffered read (which does
- * not hold the i_mutex lock) while direct IO write has
- * not completed. DIO request on holes finally falls back
- * to buffered IO for this reason.
- *
- * For ext4 extent based file, since we support fallocate,
- * new allocated extent as uninitialized, for holes, we
- * could fallocate blocks for holes, thus parallel
- * buffered IO read will zero out the page when read on
- * a hole while parallel DIO write to the hole has not completed.
- *
- * when we come here, we know it's a direct IO write to
- * to the middle of file (<i_size)
- * so it's safe to override the create flag from VFS.
- */
- create = EXT4_GET_BLOCKS_DIO_CREATE_EXT;
-
- if (max_blocks > DIO_MAX_BLOCKS)
- max_blocks = DIO_MAX_BLOCKS;
- dio_credits = ext4_chunk_trans_blocks(inode, max_blocks);
- handle = ext4_journal_start(inode, dio_credits);
- if (IS_ERR(handle)) {
- ret = PTR_ERR(handle);
- goto out;
+ * ext4_get_block in prepare for a DIO write or buffer write.
+ * We allocate an uinitialized extent if blocks haven't been allocated.
+ * The extent will be converted to initialized after IO complete.
+ */
+ create = EXT4_GET_BLOCKS_IO_CREATE_EXT;
+
+ if (!handle) {
+ if (max_blocks > DIO_MAX_BLOCKS)
+ max_blocks = DIO_MAX_BLOCKS;
+ dio_credits = ext4_chunk_trans_blocks(inode, max_blocks);
+ handle = ext4_journal_start(inode, dio_credits);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ goto out;
+ }
+ started = 1;
}
+
ret = ext4_get_blocks(handle, inode, iblock, max_blocks, bh_result,
create);
if (ret > 0) {
bh_result->b_size = (ret << inode->i_blkbits);
ret = 0;
}
- ext4_journal_stop(handle);
+ if (started)
+ ext4_journal_stop(handle);
out:
return ret;
}
@@ -3662,19 +3654,20 @@ static void ext4_free_io_end(ext4_io_end
iput(io->inode);
kfree(io);
}
-static void dump_aio_dio_list(struct inode * inode)
+
+static void dump_completed_IO(struct inode * inode)
{
#ifdef EXT4_DEBUG
struct list_head *cur, *before, *after;
ext4_io_end_t *io, *io0, *io1;

- if (list_empty(&EXT4_I(inode)->i_aio_dio_complete_list)){
- ext4_debug("inode %lu aio dio list is empty\n", inode->i_ino);
+ if (list_empty(&EXT4_I(inode)->i_completed_io_list)){
+ ext4_debug("inode %lu completed_io list is empty\n",
inode->i_ino);
return;
}

- ext4_debug("Dump inode %lu aio_dio_completed_IO list \n", inode->i_ino);
- list_for_each_entry(io, &EXT4_I(inode)->i_aio_dio_complete_list, list){
+ ext4_debug("Dump inode %lu completed_io list \n", inode->i_ino);
+ list_for_each_entry(io, &EXT4_I(inode)->i_completed_io_list, list){
cur = &io->list;
before = cur->prev;
io0 = container_of(before, ext4_io_end_t, list);
@@ -3690,21 +3683,21 @@ static void dump_aio_dio_list(struct ino
/*
* check a range of space and convert unwritten extents to written.
*/
-static int ext4_end_aio_dio_nolock(ext4_io_end_t *io)
+static int ext4_end_io_nolock(ext4_io_end_t *io)
{
struct inode *inode = io->inode;
loff_t offset = io->offset;
size_t size = io->size;
int ret = 0;

- ext4_debug("end_aio_dio_onlock: io 0x%p from inode %lu,list->next 0x%p,"
+ ext4_debug("ext4_end_io_nolock: io 0x%p from inode %lu,list->next 0x%p,"
"list->prev 0x%p\n",
io, inode->i_ino, io->list.next, io->list.prev);

if (list_empty(&io->list))
return ret;

- if (io->flag != DIO_AIO_UNWRITTEN)
+ if (io->flag != EXT4_IO_UNWRITTEN)
return ret;

if (offset + size <= i_size_read(inode))
@@ -3722,17 +3715,18 @@ static int ext4_end_aio_dio_nolock(ext4_
io->flag = 0;
return ret;
}
+
/*
* work on completed aio dio IO, to convert unwritten extents to extents
*/
-static void ext4_end_aio_dio_work(struct work_struct *work)
+static void ext4_end_io_work(struct work_struct *work)
{
ext4_io_end_t *io = container_of(work, ext4_io_end_t, work);
struct inode *inode = io->inode;
int ret = 0;

mutex_lock(&inode->i_mutex);
- ret = ext4_end_aio_dio_nolock(io);
+ ret = ext4_end_io_nolock(io);
if (ret >= 0) {
if (!list_empty(&io->list))
list_del_init(&io->list);
@@ -3740,32 +3734,35 @@ static void ext4_end_aio_dio_work(struct
}
mutex_unlock(&inode->i_mutex);
}
+
/*
* This function is called from ext4_sync_file().
*
- * When AIO DIO IO is completed, the work to convert unwritten
- * extents to written is queued on workqueue but may not get immediately
+ * When IO is completed, the work to convert unwritten extents to
+ * written is queued on workqueue but may not get immediately
* scheduled. When fsync is called, we need to ensure the
* conversion is complete before fsync returns.
- * The inode keeps track of a list of completed AIO from DIO path
- * that might needs to do the conversion. This function walks through
- * the list and convert the related unwritten extents to written.
+ * The inode keeps track of a list of pending/completed IO that
+ * might needs to do the conversion. This function walks through
+ * the list and convert the related unwritten extents for completed IO
+ * to written.
+ * The function return the number of pending IOs on success.
*/
-int flush_aio_dio_completed_IO(struct inode *inode)
+int flush_completed_IO(struct inode *inode)
{
ext4_io_end_t *io;
int ret = 0;
int ret2 = 0;

- if (list_empty(&EXT4_I(inode)->i_aio_dio_complete_list))
+ if (list_empty(&EXT4_I(inode)->i_completed_io_list))
return ret;

- dump_aio_dio_list(inode);
- while (!list_empty(&EXT4_I(inode)->i_aio_dio_complete_list)){
- io = list_entry(EXT4_I(inode)->i_aio_dio_complete_list.next,
+ dump_completed_IO(inode);
+ while (!list_empty(&EXT4_I(inode)->i_completed_io_list)){
+ io = list_entry(EXT4_I(inode)->i_completed_io_list.next,
ext4_io_end_t, list);
/*
- * Calling ext4_end_aio_dio_nolock() to convert completed
+ * Calling ext4_end_io_nolock() to convert completed
* IO to written.
*
* When ext4_sync_file() is called, run_queue() may already
@@ -3778,7 +3775,7 @@ int flush_aio_dio_completed_IO(struct in
* avoid double converting from both fsync and background work
* queue work.
*/
- ret = ext4_end_aio_dio_nolock(io);
+ ret = ext4_end_io_nolock(io);
if (ret < 0)
ret2 = ret;
else
@@ -3800,7 +3797,7 @@ static ext4_io_end_t *ext4_init_io_end (
io->offset = 0;
io->size = 0;
io->error = 0;
- INIT_WORK(&io->work, ext4_end_aio_dio_work);
+ INIT_WORK(&io->work, ext4_end_io_work);
INIT_LIST_HEAD(&io->list);
}

@@ -3823,7 +3820,7 @@ static void ext4_end_io_dio(struct kiocb
size);

/* if not aio dio with unwritten extents, just free io and return */
- if (io_end->flag != DIO_AIO_UNWRITTEN){
+ if (io_end->flag != EXT4_IO_UNWRITTEN){
ext4_free_io_end(io_end);
iocb->private = NULL;
return;
@@ -3838,9 +3835,10 @@ static void ext4_end_io_dio(struct kiocb

/* Add the io_end to per-inode completed aio dio list*/
list_add_tail(&io_end->list,
- &EXT4_I(io_end->inode)->i_aio_dio_complete_list);
+ &EXT4_I(io_end->inode)->i_completed_io_list);
iocb->private = NULL;
}
+
/*
* For ext4 extent files, ext4 will do direct-io write to holes,
* preallocated extents, and those write extend the file, no need to
@@ -3910,7 +3908,7 @@ static ssize_t ext4_ext_direct_IO(int rw
ret = blockdev_direct_IO(rw, iocb, inode,
inode->i_sb->s_bdev, iov,
offset, nr_segs,
- ext4_get_block_dio_write,
+ ext4_get_block_write,
ext4_end_io_dio);
if (iocb->private)
EXT4_I(inode)->cur_aio_dio = NULL;
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[RFC PATCH 0/4] ext4: implement DIO read nolocking

2009-12-15T17:36:23Z

Hi all,

Several weeks ago, we proposed the idea of always allocating uninitialized
extent during ext4 DIO write and/or buffer write and convert the extent into
initialized after IO complete. This allows us to drop the i_mutex lock during
ext4 DIO read, which would help improve ext4 DIO read performance with
multiple threads or AIOs running on high-speed SSDs. FYI, here is the email
thread referred to: http://marc.info/?l=linux-ext4&m=125513220225086&w=2

Following this email are our RFC patches to implement the proposed idea.
We haven't tested these patches much so they are probably very buggy.

Any feedback would be greatly appreciated!

Regards,

Jiaying
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: fix sleep inside spinlock issue aka #14739 V2

2009-12-15T15:44:10Z

On Wed 16-12-09 02:16:36, Dmitry Monakhov wrote:

> 2009/12/16 Jan Kara <jack@...>:
> > On Thu 10-12-09 20:22:16, Dmitry Monakhov wrote:
> >>
> >> drop i_block_reservation_lock before vfs_dq_reserve_block().
> >> this patch fix http://bugzilla.kernel.org/show_bug.cgi?id=14739
> >>
> >> changes from previous version:
> >> - simplify the patch according to Jan's comments
> > Dmitry, I suppose I should also merge this patch together with your other
> > fixes, right? For some reason, it was not part of the last submission of
> > your patch series...
> yes. it is not in the series because it is another bug, and it is not
> depended from
> other patches.
> Please merge newst version of the patch http://patchwork.ozlabs.org/patch/40896/
> This version is affected by stupid bug, see at the bottom of the patch.
> In fact this patch does help Justin Maggard to overcome his issue.
>
> BTW i've also have another patch http://patchwork.ozlabs.org/patch/40805/
> it was acked by you, but ASAIK it was't pushed to the ext4 queue yet.

OK, I've added both patches to my tree. Usually Ted takes care of ext4
fixes but given that I carry also other ext4 quota fixes, I think I can
merge these as well if he does not object.

Honza
--
Jan Kara <jack@...>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: fix sleep inside spinlock issue aka #14739 V2

2009-12-15T15:16:36Z

2009/12/16 Jan Kara <jack@...>:

> On Thu 10-12-09 20:22:16, Dmitry Monakhov wrote:
>>
>> drop i_block_reservation_lock before vfs_dq_reserve_block().
>> this patch fix http://bugzilla.kernel.org/show_bug.cgi?id=14739
>>
>> changes from previous version:
>> - simplify the patch according to Jan's comments
> Dmitry, I suppose I should also merge this patch together with your other
> fixes, right? For some reason, it was not part of the last submission of
> your patch series...

yes. it is not in the series because it is another bug, and it is not
depended from
other patches.
Please merge newst version of the patch http://patchwork.ozlabs.org/patch/40896/
This version is affected by stupid bug, see at the bottom of the patch.
In fact this patch does help Justin Maggard to overcome his issue.

BTW i've also have another patch http://patchwork.ozlabs.org/patch/40805/
it was acked by you, but ASAIK it was't pushed to the ext4 queue yet.

>
> Honza
>
>> Signed-off-by: Dmitry Monakhov <dmonakhov@...>
>> ---
>> fs/ext4/inode.c | 6 +++---
>> 1 files changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
>> index 942e183..2327f7a 100644
>> --- a/fs/ext4/inode.c
>> +++ b/fs/ext4/inode.c
>> @@ -1851,6 +1851,7 @@ repeat:
>>
>> md_needed = mdblocks - EXT4_I(inode)->i_reserved_meta_blocks;
>> total = md_needed + nrblocks;
>> + spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
>>
>> /*
>> * Make quota reservation here to prevent quota overflow
>> @@ -1858,12 +1859,10 @@ repeat:
>> * time.
>> */
>> if (vfs_dq_reserve_block(inode, total)) {
>> - spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
>> return -EDQUOT;
>> }
>>
>> if (ext4_claim_free_blocks(sbi, total)) {
>> - spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
>> vfs_dq_release_reservation_block(inode, total);
>> if (ext4_should_retry_alloc(inode->i_sb, &retries)) {
>> yield();
>> @@ -1871,10 +1870,11 @@ repeat:
>> }
>> return -ENOSPC;
>> }
>> + spin_lock(&EXT4_I(inode)->i_block_reservation_lock);
>> EXT4_I(inode)->i_reserved_data_blocks += nrblocks;
>> EXT4_I(inode)->i_reserved_meta_blocks = mdblocks;

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meta_blocks may be changed after we dropped the lock, so we have to
use add here:
EXT4_I(inode)->i_reserved_meta_blocks += mb_needed;
I've overlooked this simple bug, and in fact i'm able to catch it only
with havy io load
./fsstress -p16 .....

>> -
>> spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
>> +
>> return 0; /* success */
>> }
>>
>> --
>> 1.6.0.4
>>
> --
> Jan Kara <jack@...>
> SUSE Labs, CR
>

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: Ensure zeroout blocks have no dirty metadata

2009-12-15T14:33:29Z

On Fri, Dec 11, 2009 at 3:27 PM, Andreas Dilger <adilger@...> wrote:
> On 2009-12-11, at 15:01, Curt Wohlgemuth wrote:

>>
>> Good point, Andreas. I changed this to send in the handle to
>> ext4_ext_zeroout().

> I was just thinking of checking EXT4_SB(inode->i_sb)->s_journal == NULL. I
> also thought about checking EXT4_HAS_INCOMPAT_FEATURE(sb, NEEDS_RECOVERY),
> but I don't know if that is 100% safe (i.e. is it possible to mount such a
> filesystem with "norecovery"?).

Arggh. There are too many ways to check the journal use/mode... This
patch is considerably simpler.

=====================================================

This fixes a bug with no journal being used, in which new blocks returned
from an extent created with ext4_ext_zeroout() can have dirty metadata still
associated with them.

Signed-off-by: Curt Wohlgemuth <curtw@...>
---

This is for the problem I reported on 23 Nov ("Bug in extent zeroout: blocks
not marked as new"). I'm not seeing the corruption with this fix that I was
seeing without it.

diff -uprN orig/fs/ext4/extents.c new/fs/ext4/extents.c
--- orig/fs/ext4/extents.c 2009-12-09 15:09:25.000000000 -0800
+++ new/fs/ext4/extents.c 2009-12-15 13:26:29.000000000 -0800
@@ -2474,9 +2474,28 @@ static int ext4_ext_zeroout(struct inode
submit_bio(WRITE, bio);
wait_for_completion(&event);

- if (test_bit(BIO_UPTODATE, &bio->bi_flags))
+ if (test_bit(BIO_UPTODATE, &bio->bi_flags)) {
+
ret = 0;
- else {
+
+ /* On success, if there is no journal through which
+ * metadata is committed, we need to insure all
+ * metadata associated with each of these blocks is
+ * unmapped. */
+ if (EXT4_SB(inode->i_sb)->s_journal == NULL) {
+ sector_t block = ee_pblock;
+
+ done = 0;
+ while (done < len) {
+ unmap_underlying_metadata(inode->i_sb->
+ s_bdev,
+ block);
+
+ done++;
+ block++;
+ }
+ }
+ } else {
ret = -EIO;
break;
}
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[merged] tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function.patch removed from -mm tree

2009-12-15T13:57:46Z

The patch titled
tree-wide: convert open calls to remove spaces to skip_spaces() lib function
has been removed from the -mm tree. Its filename was
tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function.patch

This patch was dropped because it was merged into mainline or a subsystem tree

The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/

------------------------------------------------------
Subject: tree-wide: convert open calls to remove spaces to skip_spaces() lib function
From: André Goddard Rosa <andre.goddard@...>

Makes use of skip_spaces() defined in lib/string.c for removing leading
spaces from strings all over the tree.

It decreases lib.a code size by 47 bytes and reuses the function tree-wide:
text data bss dec hex filename
64688 584 592 65864 10148 (TOTALS-BEFORE)
64641 584 592 65817 10119 (TOTALS-AFTER)

Also, while at it, if we see (*str && isspace(*str)), we can be sure to
remove the first condition (*str) as the second one (isspace(*str)) also
evaluates to 0 whenever *str == 0, making it redundant. In other words,
"a char equals zero is never a space".

Julia Lawall tried the semantic patch (http://coccinelle.lip6.fr) below,
and found occurrences of this pattern on 3 more files:
drivers/leds/led-class.c
drivers/leds/ledtrig-timer.c
drivers/video/output.c

@@
expression str;
@@

( // ignore skip_spaces cases
while (*str && isspace(*str)) { $str++;\|++str;$ }
|
- *str &&
isspace(*str)
)

Signed-off-by: André Goddard Rosa <andre.goddard@...>
Cc: Julia Lawall <julia@...>
Cc: Martin Schwidefsky <schwidefsky@...>
Cc: Jeff Dike <jdike@...>
Cc: Ingo Molnar <mingo@...>
Cc: Thomas Gleixner <tglx@...>
Cc: "H. Peter Anvin" <hpa@...>
Cc: Richard Purdie <rpurdie@...>
Cc: Neil Brown <neilb@...>
Cc: Kyle McMartin <kyle@...>
Cc: Henrique de Moraes Holschuh <hmh@...>
Cc: David Howells <dhowells@...>
Cc: <linux-ext4@...>
Cc: Samuel Ortiz <samuel@...>
Cc: Patrick McHardy <kaber@...>
Cc: Takashi Iwai <tiwai@...>
Signed-off-by: Andrew Morton <akpm@...>
---

arch/s390/kernel/debug.c | 3 +-
arch/um/drivers/mconsole_kern.c | 16 ++++------
arch/x86/kernel/cpu/mtrr/if.c | 11 ++-----
drivers/leds/led-class.c | 2 -
drivers/leds/ledtrig-timer.c | 4 +-
drivers/md/dm-table.c | 6 +---
drivers/md/md.c | 4 +-
drivers/parisc/pdc_stable.c | 9 ++----
drivers/platform/x86/thinkpad_acpi.c | 7 +---
drivers/pnp/interface.c | 36 ++++++------------------
drivers/s390/block/dasd_proc.c | 5 ++-
drivers/video/backlight/lcd.c | 4 +-
drivers/video/display/display-sysfs.c | 2 -
drivers/video/output.c | 2 -
fs/cachefiles/daemon.c | 4 +-
fs/ext4/super.c | 7 +---
kernel/params.c | 8 ++---
lib/argv_split.c | 13 ++------
lib/dynamic_debug.c | 4 +-
lib/vsprintf.c | 15 ++--------
net/irda/irnet/irnet.h | 1
net/irda/irnet/irnet_ppp.c | 8 ++---
net/netfilter/xt_recent.c | 3 --
sound/pci/hda/hda_hwdep.c | 7 ++--
24 files changed, 66 insertions(+), 115 deletions(-)

diff -puN arch/s390/kernel/debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/s390/kernel/debug.c
--- a/arch/s390/kernel/debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/s390/kernel/debug.c
@@ -18,6 +18,7 @@
#include <linux/errno.h>
#include <linux/slab.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/sysctl.h>
#include <asm/uaccess.h>
#include <linux/module.h>
@@ -1178,7 +1179,7 @@ debug_get_uint(char *buf)
{
int rc;

- for(; isspace(*buf); buf++);
+ buf = skip_spaces(buf);
rc = simple_strtoul(buf, &buf, 10);
if(*buf){
rc = -EINVAL;
diff -puN arch/um/drivers/mconsole_kern.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/um/drivers/mconsole_kern.c
--- a/arch/um/drivers/mconsole_kern.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/um/drivers/mconsole_kern.c
@@ -6,6 +6,7 @@

#include <linux/console.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/interrupt.h>
#include <linux/list.h>
#include <linux/mm.h>
@@ -131,7 +132,7 @@ void mconsole_proc(struct mc_request *re
char *ptr = req->request.data, *buf;

ptr += strlen("proc");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);

proc = get_fs_type("proc");
if (proc == NULL) {
@@ -212,8 +213,7 @@ void mconsole_proc(struct mc_request *re
char *ptr = req->request.data;

ptr += strlen("proc");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);
snprintf(path, sizeof(path), "/proc/%s", ptr);

fd = sys_open(path, 0, 0);
@@ -560,8 +560,7 @@ void mconsole_config(struct mc_request *
int err;

ptr += strlen("config");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);
dev = mconsole_find_dev(ptr);
if (dev == NULL) {
mconsole_reply(req, "Bad configuration option", 1, 0);
@@ -588,7 +587,7 @@ void mconsole_remove(struct mc_request *
int err, start, end, n;

ptr += strlen("remove");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);
dev = mconsole_find_dev(ptr);
if (dev == NULL) {
mconsole_reply(req, "Bad remove option", 1, 0);
@@ -712,7 +711,7 @@ void mconsole_sysrq(struct mc_request *r
char *ptr = req->request.data;

ptr += strlen("sysrq");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);

/*
* With 'b', the system will shut down without a chance to reply,
@@ -757,8 +756,7 @@ void mconsole_stack(struct mc_request *r
*/

ptr += strlen("stack");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

/*
* Should really check for multiple pids or reject bad args here
diff -puN arch/x86/kernel/cpu/mtrr/if.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/x86/kernel/cpu/mtrr/if.c
--- a/arch/x86/kernel/cpu/mtrr/if.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/x86/kernel/cpu/mtrr/if.c
@@ -4,6 +4,7 @@
#include <linux/proc_fs.h>
#include <linux/module.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/init.h>

#define LINE_SIZE 80
@@ -133,8 +134,7 @@ mtrr_write(struct file *file, const char
return -EINVAL;

base = simple_strtoull(line + 5, &ptr, 0);
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

if (strncmp(ptr, "size=", 5))
return -EINVAL;
@@ -142,14 +142,11 @@ mtrr_write(struct file *file, const char
size = simple_strtoull(ptr + 5, &ptr, 0);
if ((base & 0xfff) || (size & 0xfff))
return -EINVAL;
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

if (strncmp(ptr, "type=", 5))
return -EINVAL;
- ptr += 5;
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr + 5);

for (i = 0; i < MTRR_NUM_TYPES; ++i) {
if (strcmp(ptr, mtrr_strings[i]))
diff -puN drivers/leds/led-class.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/leds/led-class.c
--- a/drivers/leds/led-class.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/leds/led-class.c
@@ -50,7 +50,7 @@ static ssize_t led_brightness_store(stru
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
diff -puN drivers/leds/ledtrig-timer.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/leds/ledtrig-timer.c
--- a/drivers/leds/ledtrig-timer.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/leds/ledtrig-timer.c
@@ -83,7 +83,7 @@ static ssize_t led_delay_on_store(struct
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
@@ -127,7 +127,7 @@ static ssize_t led_delay_off_store(struc
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
diff -puN drivers/md/dm-table.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/md/dm-table.c
--- a/drivers/md/dm-table.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/md/dm-table.c
@@ -12,6 +12,7 @@
#include <linux/blkdev.h>
#include <linux/namei.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/mutex.h>
@@ -600,11 +601,8 @@ int dm_split_args(int *argc, char ***arg
return -ENOMEM;

while (1) {
- start = end;
-
/* Skip whitespace */
- while (*start && isspace(*start))
- start++;
+ start = skip_spaces(end);

if (!*start)
break; /* success, we hit the end */
diff -puN drivers/md/md.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/md/md.c
--- a/drivers/md/md.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/md/md.c
@@ -39,6 +39,7 @@
#include <linux/buffer_head.h> /* for invalidate_bdev */
#include <linux/poll.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/hdreg.h>
#include <linux/proc_fs.h>
#include <linux/random.h>
@@ -3246,8 +3247,7 @@ bitmap_store(mddev_t *mddev, const char
}
if (*end && !isspace(*end)) break;
bitmap_dirty_bits(mddev->bitmap, chunk, end_chunk);
- buf = end;
- while (isspace(*buf)) buf++;
+ buf = skip_spaces(end);
}
bitmap_unplug(mddev->bitmap); /* flush the bits to disk */
out:
diff -puN drivers/parisc/pdc_stable.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/parisc/pdc_stable.c
--- a/drivers/parisc/pdc_stable.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/parisc/pdc_stable.c
@@ -779,12 +779,9 @@ static ssize_t pdcs_auto_write(struct ko
read_unlock(&pathentry->rw_lock);

DPRINTK("%s: flags before: 0x%X\n", __func__, flags);
-
- temp = in;
-
- while (*temp && isspace(*temp))
- temp++;
-
+
+ temp = skip_spaces(in);
+
c = *temp++ - '0';
if ((c != 0) && (c != 1))
goto parse_error;
diff -puN drivers/platform/x86/thinkpad_acpi.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/platform/x86/thinkpad_acpi.c
--- a/drivers/platform/x86/thinkpad_acpi.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/platform/x86/thinkpad_acpi.c
@@ -1006,11 +1006,8 @@ static int parse_strtoul(const char *buf
{
char *endp;

- while (*buf && isspace(*buf))
- buf++;
- *value = simple_strtoul(buf, &endp, 0);
- while (*endp && isspace(*endp))
- endp++;
+ *value = simple_strtoul(skip_spaces(buf), &endp, 0);
+ endp = skip_spaces(endp);
if (*endp || *value > max)
return -EINVAL;

diff -puN drivers/pnp/interface.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/pnp/interface.c
--- a/drivers/pnp/interface.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/pnp/interface.c
@@ -310,8 +310,7 @@ static ssize_t pnp_set_current_resources
goto done;
}

- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (!strnicmp(buf, "disable", 7)) {
retval = pnp_disable_dev(dev);
goto done;
@@ -353,19 +352,13 @@ static ssize_t pnp_set_current_resources
pnp_init_resources(dev);
mutex_lock(&pnp_res_mutex);
while (1) {
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (!strnicmp(buf, "io", 2)) {
- buf += 2;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 2);
start = simple_strtoul(buf, &buf, 0);
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (*buf == '-') {
- buf += 1;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 1);
end = simple_strtoul(buf, &buf, 0);
} else
end = start;
@@ -373,16 +366,11 @@ static ssize_t pnp_set_current_resources
continue;
}
if (!strnicmp(buf, "mem", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (*buf == '-') {
- buf += 1;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 1);
end = simple_strtoul(buf, &buf, 0);
} else
end = start;
@@ -390,17 +378,13 @@ static ssize_t pnp_set_current_resources
continue;
}
if (!strnicmp(buf, "irq", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
pnp_add_irq_resource(dev, start, 0);
continue;
}
if (!strnicmp(buf, "dma", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
pnp_add_dma_resource(dev, start, 0);
continue;
diff -puN drivers/s390/block/dasd_proc.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/s390/block/dasd_proc.c
--- a/drivers/s390/block/dasd_proc.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/s390/block/dasd_proc.c
@@ -14,6 +14,7 @@
#define KMSG_COMPONENT "dasd"

#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/seq_file.h>
#include <linux/vmalloc.h>
#include <linux/proc_fs.h>
@@ -272,10 +273,10 @@ dasd_statistics_write(struct file *file,
DBF_EVENT(DBF_DEBUG, "/proc/dasd/statictics: '%s'\n", buffer);

/* check for valid verbs */
- for (str = buffer; isspace(*str); str++);
+ str = skip_spaces(buffer);
if (strncmp(str, "set", 3) == 0 && isspace(str[3])) {
/* 'set xxx' was given */
- for (str = str + 4; isspace(*str); str++);
+ str = skip_spaces(str + 4);
if (strcmp(str, "on") == 0) {
/* switch on statistics profiling */
dasd_profile_level = DASD_PROFILE_ON;
diff -puN drivers/video/backlight/lcd.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/backlight/lcd.c
--- a/drivers/video/backlight/lcd.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/backlight/lcd.c
@@ -101,7 +101,7 @@ static ssize_t lcd_store_power(struct de
int power = simple_strtoul(buf, &endp, 0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
@@ -140,7 +140,7 @@ static ssize_t lcd_store_contrast(struct
int contrast = simple_strtoul(buf, &endp, 0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
diff -puN drivers/video/display/display-sysfs.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/display/display-sysfs.c
--- a/drivers/video/display/display-sysfs.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/display/display-sysfs.c
@@ -67,7 +67,7 @@ static ssize_t display_store_contrast(st
contrast = simple_strtoul(buf, &endp, 0);
size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;

if (size != count)
diff -puN drivers/video/output.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/output.c
--- a/drivers/video/output.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/output.c
@@ -50,7 +50,7 @@ static ssize_t video_output_store_state(
int request_state = simple_strtoul(buf,&endp,0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
diff -puN fs/cachefiles/daemon.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function fs/cachefiles/daemon.c
--- a/fs/cachefiles/daemon.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/fs/cachefiles/daemon.c
@@ -21,6 +21,7 @@
#include <linux/mount.h>
#include <linux/statfs.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/fs_struct.h>
#include "internal.h"

@@ -257,8 +258,7 @@ static ssize_t cachefiles_daemon_write(s
if (args == data)
goto error;
*args = '\0';
- for (args++; isspace(*args); args++)
- continue;
+ args = skip_spaces(++args);
}

/* run the appropriate command handler */
diff -puN fs/ext4/super.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function fs/ext4/super.c
--- a/fs/ext4/super.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/fs/ext4/super.c
@@ -2137,11 +2137,8 @@ static int parse_strtoul(const char *buf
{
char *endp;

- while (*buf && isspace(*buf))
- buf++;
- *value = simple_strtoul(buf, &endp, 0);
- while (*endp && isspace(*endp))
- endp++;
+ *value = simple_strtoul(skip_spaces(buf), &endp, 0);
+ endp = skip_spaces(endp);
if (*endp || *value > max)
return -EINVAL;

diff -puN kernel/params.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function kernel/params.c
--- a/kernel/params.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/kernel/params.c
@@ -24,6 +24,7 @@
#include <linux/err.h>
#include <linux/slab.h>
#include <linux/ctype.h>
+#include <linux/string.h>

#if 0
#define DEBUGP printk
@@ -122,9 +123,7 @@ static char *next_arg(char *args, char *
next = args + i;

/* Chew up trailing spaces. */
- while (isspace(*next))
- next++;
- return next;
+ return skip_spaces(next);
}

/* Args looks like "foo=bar,bar2 baz=fuz wiz". */
@@ -139,8 +138,7 @@ int parse_args(const char *name,
DEBUGP("Parsing ARGS: %s\n", args);

/* Chew leading spaces */
- while (isspace(*args))
- args++;
+ args = skip_spaces(args);

while (*args) {
int ret;
diff -puN lib/argv_split.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/argv_split.c
--- a/lib/argv_split.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/argv_split.c
@@ -4,17 +4,10 @@

#include <linux/kernel.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/slab.h>
#include <linux/module.h>

-static const char *skip_sep(const char *cp)
-{
- while (*cp && isspace(*cp))
- cp++;
-
- return cp;
-}
-
static const char *skip_arg(const char *cp)
{
while (*cp && !isspace(*cp))
@@ -28,7 +21,7 @@ static int count_argc(const char *str)
int count = 0;

while (*str) {
- str = skip_sep(str);
+ str = skip_spaces(str);
if (*str) {
count++;
str = skip_arg(str);
@@ -82,7 +75,7 @@ char **argv_split(gfp_t gfp, const char
argvp = argv;

while (*str) {
- str = skip_sep(str);
+ str = skip_spaces(str);

if (*str) {
const char *p = str;
diff -puN lib/dynamic_debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/dynamic_debug.c
--- a/lib/dynamic_debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/dynamic_debug.c
@@ -21,6 +21,7 @@
#include <linux/list.h>
#include <linux/sysctl.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/dynamic_debug.h>
#include <linux/debugfs.h>
@@ -209,8 +210,7 @@ static int ddebug_tokenize(char *buf, ch
char *end;

/* Skip leading whitespace */
- while (*buf && isspace(*buf))
- buf++;
+ buf = skip_spaces(buf);
if (!*buf)
break; /* oh, it was trailing whitespace */

diff -puN lib/vsprintf.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/vsprintf.c
--- a/lib/vsprintf.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/vsprintf.c
@@ -1766,13 +1766,6 @@ EXPORT_SYMBOL_GPL(bprintf);

#endif /* CONFIG_BINARY_PRINTF */

-static noinline char *skip_space(const char *str)
-{
- while (isspace(*str))
- ++str;
- return (char *)str;
-}
-
/**
* vsscanf - Unformat a buffer into a list of arguments
* @buf: input buffer
@@ -1794,8 +1787,8 @@ int vsscanf(const char *buf, const char
* white space, including none, in the input.
*/
if (isspace(*fmt)) {
- fmt = skip_space(fmt);
- str = skip_space(str);
+ fmt = skip_spaces(++fmt);
+ str = skip_spaces(str);
}

/* anything that is not a conversion must match exactly */
@@ -1865,7 +1858,7 @@ int vsscanf(const char *buf, const char
if (field_width == -1)
field_width = INT_MAX;
/* first, skip leading white space in buffer */
- str = skip_space(str);
+ str = skip_spaces(str);

/* now copy until next white space */
while (*str && !isspace(*str) && field_width--)
@@ -1907,7 +1900,7 @@ int vsscanf(const char *buf, const char
/* have some sort of integer conversion.
* first, skip white space in buffer.
*/
- str = skip_space(str);
+ str = skip_spaces(str);

digit = *str;
if (is_sign && digit == '-')
diff -puN net/irda/irnet/irnet.h~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/irda/irnet/irnet.h
--- a/net/irda/irnet/irnet.h~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/irda/irnet/irnet.h
@@ -249,6 +249,7 @@
#include <linux/poll.h>
#include <linux/capability.h>
#include <linux/ctype.h> /* isspace() */
+#include <linux/string.h> /* skip_spaces() */
#include <asm/uaccess.h>
#include <linux/init.h>

diff -puN net/irda/irnet/irnet_ppp.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/irda/irnet/irnet_ppp.c
--- a/net/irda/irnet/irnet_ppp.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/irda/irnet/irnet_ppp.c
@@ -76,9 +76,8 @@ irnet_ctrl_write(irnet_socket * ap,
/* Look at the next command */
start = next;

- /* Scrap whitespaces before the command */
- while(isspace(*start))
- start++;
+ /* Scrap whitespaces before the command */
+ start = skip_spaces(start);

/* ',' is our command separator */
next = strchr(start, ',');
@@ -133,8 +132,7 @@ irnet_ctrl_write(irnet_socket * ap,
char * endp;

/* Scrap whitespaces before the command */
- while(isspace(*begp))
- begp++;
+ begp = skip_spaces(begp);

/* Convert argument to a number (last arg is the base) */
addr = simple_strtoul(begp, &endp, 16);
diff -puN net/netfilter/xt_recent.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/netfilter/xt_recent.c
--- a/net/netfilter/xt_recent.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/netfilter/xt_recent.c
@@ -482,8 +482,7 @@ static ssize_t recent_old_proc_write(str
if (copy_from_user(buf, input, size))
return -EFAULT;

- while (isspace(*c))
- c++;
+ c = skip_spaces(c);

if (size - (c - buf) < 5)
return c - buf;
diff -puN sound/pci/hda/hda_hwdep.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function sound/pci/hda/hda_hwdep.c
--- a/sound/pci/hda/hda_hwdep.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/sound/pci/hda/hda_hwdep.c
@@ -24,6 +24,7 @@
#include <linux/compat.h>
#include <linux/mutex.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/firmware.h>
#include <sound/core.h>
#include "hda_codec.h"
@@ -428,8 +429,7 @@ static int parse_hints(struct hda_codec
char *key, *val;
struct hda_hint *hint;

- while (isspace(*buf))
- buf++;
+ buf = skip_spaces(buf);
if (!*buf || *buf == '#' || *buf == '\n')
return 0;
if (*buf == '=')
@@ -444,8 +444,7 @@ static int parse_hints(struct hda_codec
return -EINVAL;
}
*val++ = 0;
- while (isspace(*val))
- val++;
+ val = skip_spaces(val);
remove_trail_spaces(key);
remove_trail_spaces(val);
hint = get_hint(codec, key);
_

Patches currently in -mm which might be from andre.goddard@... are

origin.patch
pid-tighten-pidmap-spinlock-critical-section-by-removing-kfree.patch
pid-reduce-code-size-by-using-a-pointer-to-iterate-over-array.patch

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] ext4: fix sleep inside spinlock issue aka #14739 V2

2009-12-15T13:48:06Z

On Thu 10-12-09 20:22:16, Dmitry Monakhov wrote:
>
> drop i_block_reservation_lock before vfs_dq_reserve_block().
> this patch fix http://bugzilla.kernel.org/show_bug.cgi?id=14739
>
> changes from previous version:
> - simplify the patch according to Jan's comments
Dmitry, I suppose I should also merge this patch together with your other
fixes, right? For some reason, it was not part of the last submission of
your patch series...

Honza

> Signed-off-by: Dmitry Monakhov <dmonakhov@...>
> ---
> fs/ext4/inode.c | 6 +++---
> 1 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 942e183..2327f7a 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -1851,6 +1851,7 @@ repeat:
>
> md_needed = mdblocks - EXT4_I(inode)->i_reserved_meta_blocks;
> total = md_needed + nrblocks;
> + spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
>
> /*
> * Make quota reservation here to prevent quota overflow
> @@ -1858,12 +1859,10 @@ repeat:
> * time.
> */
> if (vfs_dq_reserve_block(inode, total)) {
> - spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
> return -EDQUOT;
> }
>
> if (ext4_claim_free_blocks(sbi, total)) {
> - spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
> vfs_dq_release_reservation_block(inode, total);
> if (ext4_should_retry_alloc(inode->i_sb, &retries)) {
> yield();
> @@ -1871,10 +1870,11 @@ repeat:
> }
> return -ENOSPC;
> }
> + spin_lock(&EXT4_I(inode)->i_block_reservation_lock);
> EXT4_I(inode)->i_reserved_data_blocks += nrblocks;
> EXT4_I(inode)->i_reserved_meta_blocks = mdblocks;
> -
> spin_unlock(&EXT4_I(inode)->i_block_reservation_lock);
> +
> return 0; /* success */
> }
>
> --
> 1.6.0.4
>

Re: [PATCH] Possible data loss on ext[34], reiserfs with external journal

2009-12-15T12:01:32Z

Hello!

On Dec 15, 2009, at 11:45 AM, tytso@... wrote:

>>> + /*
>>> + * If the journal is not located on the file system device,
>>> + * then we must flush the file system device before we issue
>>> + * the commit record
>>> + */
>>> + if (commit_transaction->t_flushed_data_blocks &&
>>> + (journal->j_fs_dev != journal->j_dev) &&
>>> + (journal->j_flags & JBD2_BARRIER))
>>> + blkdev_issue_flush(journal->j_fs_dev, NULL);
>>> +
>> I am afraid this is not enough. This code is called after journal
>> was flushed for async commit case, so it leaves a race window where
>> journal transaction is already on disk and complete, but the data is
>> still in cache somewhere.
> No, that's actually fine. In the ASYNC_COMMIT case, the commit won't
> be valid until the checksum is correct, and we won't have written any
> descriptor blocks yet at this point. So there is no race because

What do you mean by descriptor blocks? Actual logged blocks?

> during that window, the commit is written but we won't write any
> descriptor blocks until after the barrier returns.

From my reading of the code, in jbd2_journal_commit_transaction:
After first "JBD: commit phase 2" message we submit actual data buffers.
Then after second "JBD: commit phase 2" message (in the
"while (commit_transaction->t_buffers) {" loop) we actually iterate
through all metadata changes and submit them to disk (after
start_journal_io: label)
Then journal_submit_commit_record writes commit record.
At this point on we have entire transacton + commit
block in the write queue (or partially already on disk) +
all the data blocks in the write queue.
Next blkdev_issue_flush on journal device is done, cementing the transaction
on the permanent storage, but the actual data is still potentially not
there abd it is not until the call to journal_finish_inode_data_buffers()
where that data is flushed too.

If you refer to the code after "JBD: commit phase 3" that waits on actual
transaction blocks buffers to make sure they made it to the disk,
my reading of blkdev_issue_flush is that it adds the flush at the end of the
queue, i.e. after all of the journalled blocks that are sitting in there,
so they all would be already copleted by the time blkdev_issue_flush returns.

Bye,
Oleg--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Possible data loss on ext[34], reiserfs with external journal

2009-12-15T08:45:03Z

On Tue, Dec 15, 2009 at 01:19:57AM -0500, Oleg Drokin wrote:

> > + /*
> > + * If the journal is not located on the file system device,
> > + * then we must flush the file system device before we issue
> > + * the commit record
> > + */
> > + if (commit_transaction->t_flushed_data_blocks &&
> > + (journal->j_fs_dev != journal->j_dev) &&
> > + (journal->j_flags & JBD2_BARRIER))
> > + blkdev_issue_flush(journal->j_fs_dev, NULL);
> > +
>
> I am afraid this is not enough. This code is called after journal
> was flushed for async commit case, so it leaves a race window where
> journal transaction is already on disk and complete, but the data is
> still in cache somewhere.

No, that's actually fine. In the ASYNC_COMMIT case, the commit won't
be valid until the checksum is correct, and we won't have written any
descriptor blocks yet at this point. So there is no race because
during that window, the commit is written but we won't write any
descriptor blocks until after the barrier returns.

> Also the callsite has this comment which is misleading, I think:
> /*
> * This is the right place to wait for data buffers both for ASYNC
> * and !ASYNC commit. If commit is ASYNC, we need to wait only after
> * the commit block went to disk (which happens above). If commit is
> * SYNC, we need to wait for data buffers before we start writing
> * commit block, which happens below in such setting.
> */

Yeah, that comment is confusing and not entirely accurate. I thought
about cleaning it up, and then decided to do that in a separate patch.

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Bug 14256] kernel BUG at fs/ext3/super.c:435

2009-12-15T01:08:48Z

http://bugzilla.kernel.org/show_bug.cgi?id=14256

--- Comment #16 from Mikael Pettersson <mikpe@...> 2009-12-15 09:08:45 ---
The affected file system is /tmp. There's nothing special about it, except it
being separate from / and /home. The builds are plain "rpmbuild --rebuild" so
they do most accesses under /usr/src/ and /var/tmp/, both on /, but the glibc
test suite will create temporary files on /tmp. I have no idea what those files
are.

--
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Possible data loss on ext[34], reiserfs with external journal

2009-12-14T22:19:57Z

Hello!

On Dec 15, 2009, at 12:32 AM, tytso@... wrote:
> Can you separate this patch into separate ones for each file system?

Sure.

> I think we can actually do better for ext4; for example, in the case
> of data=journal or data=writeback, it's not necessary to flush the fs
> data device. It's only the case of data=ordered that we need to send
> a barrier. With that optimization, we do need to add a barrier in the
> case of fsync() and when we are doing a journal checkpoint, but the
> extra code complexity is worth not having to force a barrier for the
> fs data device for every single commit, especially in the data=journal
> mode with a heavy fsync workload.

Indeed, this is a good idea too.
I think we still can squeeze a bit more juice out of it if we can
submit the data device flush right after submitting all the data, but
only do the waiting for it before issuing journal device flush in normal
journaling mode (we need to do the waiting before writing commit block
in async journaling mode).

> Do you have a test case that will allow you to easily try out this
> patch, in all of ext4's various journalling modes? Thanks!!

Not for vanilla kernel, but I'll see if I can construct something easily
for it.

> @@ -277,6 +278,16 @@ static int journal_finish_inode_data_buffers(journal_t *journal,
> struct jbd2_inode *jinode, *next_i;
> int err, ret = 0;
>
> + /*
> + * If the journal is not located on the file system device,
> + * then we must flush the file system device before we issue
> + * the commit record
> + */
> + if (commit_transaction->t_flushed_data_blocks &&
> + (journal->j_fs_dev != journal->j_dev) &&
> + (journal->j_flags & JBD2_BARRIER))
> + blkdev_issue_flush(journal->j_fs_dev, NULL);
> +

I am afraid this is not enough. This code is called after journal was flushed for
async commit case, so it leaves a race window where journal transaction is already
on disk and complete, but the data is still in cache somewhere.

Also the callsite has this comment which is misleading, I think:
/*
* This is the right place to wait for data buffers both for ASYNC
* and !ASYNC commit. If commit is ASYNC, we need to wait only after
* the commit block went to disk (which happens above). If commit is
* SYNC, we need to wait for data buffers before we start writing
* commit block, which happens below in such setting.
*/

In fact it is only safe to wait here in ASYNC mode (and internal journal only) because
the device was already flushed (and I hope all device drivers drain the queue too, if not,
even this might be problematic) by the blkdev_issue_flush.

Bye,
Oleg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Possible data loss on ext[34], reiserfs with external journal

2009-12-14T21:32:07Z

On Fri, Dec 11, 2009 at 11:16:08AM +0300, Oleg Drokin wrote:

> Hello!
>
> It seems when external journal device is used for ext3, ext4 and reiserfs
> (possibly others, but I can only readily confirm these three) and
> main filesystem device had writeback cache enabled, a very real
> data loss is possible because we never flush main device cache on commit.
> As a result if we just wrote some files in ordered mode and then
> transaction was committed, the journal data makes it to the disk
> (provided that barriers are in use), but the actual file data only made
> it to the device cache. As such sudden loss of power at this stage
> would lead to files in place, but their content replaced with
> whatever happened to be in those blocks before.
>
> This simple patch at the end should remedy the problem.

Can you separate this patch into separate ones for each file system?
I think we can actually do better for ext4; for example, in the case
of data=journal or data=writeback, it's not necessary to flush the fs
data device. It's only the case of data=ordered that we need to send
a barrier. With that optimization, we do need to add a barrier in the
case of fsync() and when we are doing a journal checkpoint, but the
extra code complexity is worth not having to force a barrier for the
fs data device for every single commit, especially in the data=journal
mode with a heavy fsync workload.

Do you have a test case that will allow you to easily try out this
patch, in all of ext4's various journalling modes? Thanks!!

- Ted

commit 4ba96b277f26952097d15736f6e960bc84cf4c0c
Author: Theodore Ts'o <tytso@...>
Date: Tue Dec 15 00:31:12 2009 -0500

ext4, jbd2: Add barriers for file systems with exernal journals

This is a bit complicated because we are trying to optimize when we
send barriers to the fs data disk. We could just throw in an extra
barrier to the data disk whenever we send a barrier to the journal
disk, but that's not always strictly necessary.

We only need to send a barrier during a commit when there are data
blocks which are must be written out due to an inode written in
ordered mode, or if fsync() depends on the commit to force data blocks
to disk. Finally, before we drop transactions from the beginning of
the journal during a checkpoint operation, we need to guarantee that
any blocks that were flushed out to the data disk are firmly on the
rust platter before we drop the transaction from the journal.

Thanks to Oleg Drokin for pointing out this flaw in ext3/ext4.

Signed-off-by: "Theodore Ts'o" <tytso@...>

diff --git a/fs/ext4/fsync.c b/fs/ext4/fsync.c
index 0b22497..98bd140 100644
--- a/fs/ext4/fsync.c
+++ b/fs/ext4/fsync.c
@@ -88,9 +88,21 @@ int ext4_sync_file(struct file *file, struct dentry *dentry, int datasync)
return ext4_force_commit(inode->i_sb);

commit_tid = datasync ? ei->i_datasync_tid : ei->i_sync_tid;
- if (jbd2_log_start_commit(journal, commit_tid))
+ if (jbd2_log_start_commit(journal, commit_tid)) {
+ /*
+ * When the journal is on a different device than the
+ * fs data disk, we need to issue the barrier in
+ * writeback mode. (In ordered mode, the jbd2 layer
+ * will take care of issuing the barrier. In
+ * data=journal, all of the data blocks are written to
+ * the journal device.)
+ */
+ if (ext4_should_writeback_data(inode) &&
+ (journal->j_fs_dev != journal->j_dev) &&
+ (journal->j_flags & JBD2_BARRIER))
+ blkdev_issue_flush(inode->i_sb->s_bdev, NULL);
jbd2_log_wait_commit(journal, commit_tid);
- else if (journal->j_flags & JBD2_BARRIER)
+ } else if (journal->j_flags & JBD2_BARRIER)
blkdev_issue_flush(inode->i_sb->s_bdev, NULL);
return ret;
}
diff --git a/fs/jbd2/checkpoint.c b/fs/jbd2/checkpoint.c
index ca0f5eb..8868493 100644
--- a/fs/jbd2/checkpoint.c
+++ b/fs/jbd2/checkpoint.c
@@ -22,6 +22,7 @@
#include <linux/jbd2.h>
#include <linux/errno.h>
#include <linux/slab.h>
+#include <linux/blkdev.h>
#include <trace/events/jbd2.h>

/*
@@ -515,6 +516,20 @@ int jbd2_cleanup_journal_tail(journal_t *journal)
journal->j_tail_sequence = first_tid;
journal->j_tail = blocknr;
spin_unlock(&journal->j_state_lock);
+
+ /*
+ * If there is an external journal, we need to make sure that
+ * any data blocks that were recently written out --- perhaps
+ * by jbd2_log_do_checkpoint() --- are flushed out before we
+ * drop the transactions from the external journal. It's
+ * unlikely this will be necessary, especially with a
+ * appropriately sized journal, but we need this to guarantee
+ * correctness. Fortunately jbd2_cleanup_journal_tail()
+ * doesn't get called all that often.
+ */
+ if ((journal->j_fs_dev != journal->j_dev) &&
+ (journal->j_flags & JBD2_BARRIER))
+ blkdev_issue_flush(journal->j_fs_dev, NULL);
if (!(journal->j_flags & JBD2_ABORT))
jbd2_journal_update_superblock(journal, 1);
return 0;
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
index 6a10238..5b78f9a 100644
--- a/fs/jbd2/commit.c
+++ b/fs/jbd2/commit.c
@@ -259,6 +259,7 @@ static int journal_submit_data_buffers(journal_t *journal,
ret = err;
spin_lock(&journal->j_list_lock);
J_ASSERT(jinode->i_transaction == commit_transaction);
+ commit_transaction->t_flushed_data_blocks = 1;
jinode->i_flags &= ~JI_COMMIT_RUNNING;
wake_up_bit(&jinode->i_flags, __JI_COMMIT_RUNNING);
}
@@ -277,6 +278,16 @@ static int journal_finish_inode_data_buffers(journal_t *journal,
struct jbd2_inode *jinode, *next_i;
int err, ret = 0;

+ /*
+ * If the journal is not located on the file system device,
+ * then we must flush the file system device before we issue
+ * the commit record
+ */
+ if (commit_transaction->t_flushed_data_blocks &&
+ (journal->j_fs_dev != journal->j_dev) &&
+ (journal->j_flags & JBD2_BARRIER))
+ blkdev_issue_flush(journal->j_fs_dev, NULL);
+
/* For locking, see the comment in journal_submit_data_buffers() */
spin_lock(&journal->j_list_lock);
list_for_each_entry(jinode, &commit_transaction->t_inode_list, i_list) {
diff --git a/include/linux/jbd2.h b/include/linux/jbd2.h
index f1011f7..638ce45 100644
--- a/include/linux/jbd2.h
+++ b/include/linux/jbd2.h
@@ -653,6 +653,7 @@ struct transaction_s
* waiting for it to finish.
*/
unsigned int t_synchronous_commit:1;
+ unsigned int t_flushed_data_blocks:1;

/*
* For use by the filesystem to store fs-specific data
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[patch 156/196] tree-wide: convert open calls to remove spaces to skip_spaces() lib function

2009-12-14T18:01:06Z

From: André Goddard Rosa <andre.goddard@...>

Makes use of skip_spaces() defined in lib/string.c for removing leading
spaces from strings all over the tree.

It decreases lib.a code size by 47 bytes and reuses the function tree-wide:
text data bss dec hex filename
64688 584 592 65864 10148 (TOTALS-BEFORE)
64641 584 592 65817 10119 (TOTALS-AFTER)

Also, while at it, if we see (*str && isspace(*str)), we can be sure to
remove the first condition (*str) as the second one (isspace(*str)) also
evaluates to 0 whenever *str == 0, making it redundant. In other words,
"a char equals zero is never a space".

Julia Lawall tried the semantic patch (http://coccinelle.lip6.fr) below,
and found occurrences of this pattern on 3 more files:
drivers/leds/led-class.c
drivers/leds/ledtrig-timer.c
drivers/video/output.c

@@
expression str;
@@

( // ignore skip_spaces cases
while (*str && isspace(*str)) { $str++;\|++str;$ }
|
- *str &&
isspace(*str)
)

Signed-off-by: André Goddard Rosa <andre.goddard@...>
Cc: Julia Lawall <julia@...>
Cc: Martin Schwidefsky <schwidefsky@...>
Cc: Jeff Dike <jdike@...>
Cc: Ingo Molnar <mingo@...>
Cc: Thomas Gleixner <tglx@...>
Cc: "H. Peter Anvin" <hpa@...>
Cc: Richard Purdie <rpurdie@...>
Cc: Neil Brown <neilb@...>
Cc: Kyle McMartin <kyle@...>
Cc: Henrique de Moraes Holschuh <hmh@...>
Cc: David Howells <dhowells@...>
Cc: <linux-ext4@...>
Cc: Samuel Ortiz <samuel@...>
Cc: Patrick McHardy <kaber@...>
Cc: Takashi Iwai <tiwai@...>
Signed-off-by: Andrew Morton <akpm@...>
---

arch/s390/kernel/debug.c | 3 +-
arch/um/drivers/mconsole_kern.c | 16 ++++------
arch/x86/kernel/cpu/mtrr/if.c | 11 ++-----
drivers/leds/led-class.c | 2 -
drivers/leds/ledtrig-timer.c | 4 +-
drivers/md/dm-table.c | 6 +---
drivers/md/md.c | 4 +-
drivers/parisc/pdc_stable.c | 9 ++----
drivers/platform/x86/thinkpad_acpi.c | 7 +---
drivers/pnp/interface.c | 36 ++++++------------------
drivers/s390/block/dasd_proc.c | 5 ++-
drivers/video/backlight/lcd.c | 4 +-
drivers/video/display/display-sysfs.c | 2 -
drivers/video/output.c | 2 -
fs/cachefiles/daemon.c | 4 +-
fs/ext4/super.c | 7 +---
kernel/params.c | 8 ++---
lib/argv_split.c | 13 ++------
lib/dynamic_debug.c | 4 +-
lib/vsprintf.c | 15 ++--------
net/irda/irnet/irnet.h | 1
net/irda/irnet/irnet_ppp.c | 8 ++---
net/netfilter/xt_recent.c | 3 --
sound/pci/hda/hda_hwdep.c | 7 ++--
24 files changed, 66 insertions(+), 115 deletions(-)

diff -puN arch/s390/kernel/debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/s390/kernel/debug.c
--- a/arch/s390/kernel/debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/s390/kernel/debug.c
@@ -18,6 +18,7 @@
#include <linux/errno.h>
#include <linux/slab.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/sysctl.h>
#include <asm/uaccess.h>
#include <linux/module.h>
@@ -1178,7 +1179,7 @@ debug_get_uint(char *buf)
{
int rc;

- for(; isspace(*buf); buf++);
+ buf = skip_spaces(buf);
rc = simple_strtoul(buf, &buf, 10);
if(*buf){
rc = -EINVAL;
diff -puN arch/um/drivers/mconsole_kern.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/um/drivers/mconsole_kern.c
--- a/arch/um/drivers/mconsole_kern.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/um/drivers/mconsole_kern.c
@@ -6,6 +6,7 @@

#include <linux/console.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/interrupt.h>
#include <linux/list.h>
#include <linux/mm.h>
@@ -131,7 +132,7 @@ void mconsole_proc(struct mc_request *re
char *ptr = req->request.data, *buf;

ptr += strlen("proc");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);

proc = get_fs_type("proc");
if (proc == NULL) {
@@ -212,8 +213,7 @@ void mconsole_proc(struct mc_request *re
char *ptr = req->request.data;

ptr += strlen("proc");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);
snprintf(path, sizeof(path), "/proc/%s", ptr);

fd = sys_open(path, 0, 0);
@@ -560,8 +560,7 @@ void mconsole_config(struct mc_request *
int err;

ptr += strlen("config");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);
dev = mconsole_find_dev(ptr);
if (dev == NULL) {
mconsole_reply(req, "Bad configuration option", 1, 0);
@@ -588,7 +587,7 @@ void mconsole_remove(struct mc_request *
int err, start, end, n;

ptr += strlen("remove");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);
dev = mconsole_find_dev(ptr);
if (dev == NULL) {
mconsole_reply(req, "Bad remove option", 1, 0);
@@ -712,7 +711,7 @@ void mconsole_sysrq(struct mc_request *r
char *ptr = req->request.data;

ptr += strlen("sysrq");
- while (isspace(*ptr)) ptr++;
+ ptr = skip_spaces(ptr);

/*
* With 'b', the system will shut down without a chance to reply,
@@ -757,8 +756,7 @@ void mconsole_stack(struct mc_request *r
*/

ptr += strlen("stack");
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

/*
* Should really check for multiple pids or reject bad args here
diff -puN arch/x86/kernel/cpu/mtrr/if.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function arch/x86/kernel/cpu/mtrr/if.c
--- a/arch/x86/kernel/cpu/mtrr/if.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/arch/x86/kernel/cpu/mtrr/if.c
@@ -4,6 +4,7 @@
#include <linux/proc_fs.h>
#include <linux/module.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/init.h>

#define LINE_SIZE 80
@@ -133,8 +134,7 @@ mtrr_write(struct file *file, const char
return -EINVAL;

base = simple_strtoull(line + 5, &ptr, 0);
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

if (strncmp(ptr, "size=", 5))
return -EINVAL;
@@ -142,14 +142,11 @@ mtrr_write(struct file *file, const char
size = simple_strtoull(ptr + 5, &ptr, 0);
if ((base & 0xfff) || (size & 0xfff))
return -EINVAL;
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr);

if (strncmp(ptr, "type=", 5))
return -EINVAL;
- ptr += 5;
- while (isspace(*ptr))
- ptr++;
+ ptr = skip_spaces(ptr + 5);

for (i = 0; i < MTRR_NUM_TYPES; ++i) {
if (strcmp(ptr, mtrr_strings[i]))
diff -puN drivers/leds/led-class.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/leds/led-class.c
--- a/drivers/leds/led-class.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/leds/led-class.c
@@ -50,7 +50,7 @@ static ssize_t led_brightness_store(stru
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
diff -puN drivers/leds/ledtrig-timer.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/leds/ledtrig-timer.c
--- a/drivers/leds/ledtrig-timer.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/leds/ledtrig-timer.c
@@ -83,7 +83,7 @@ static ssize_t led_delay_on_store(struct
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
@@ -127,7 +127,7 @@ static ssize_t led_delay_off_store(struc
unsigned long state = simple_strtoul(buf, &after, 10);
size_t count = after - buf;

- if (*after && isspace(*after))
+ if (isspace(*after))
count++;

if (count == size) {
diff -puN drivers/md/dm-table.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/md/dm-table.c
--- a/drivers/md/dm-table.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/md/dm-table.c
@@ -12,6 +12,7 @@
#include <linux/blkdev.h>
#include <linux/namei.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/mutex.h>
@@ -600,11 +601,8 @@ int dm_split_args(int *argc, char ***arg
return -ENOMEM;

while (1) {
- start = end;
-
/* Skip whitespace */
- while (*start && isspace(*start))
- start++;
+ start = skip_spaces(end);

if (!*start)
break; /* success, we hit the end */
diff -puN drivers/md/md.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/md/md.c
--- a/drivers/md/md.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/md/md.c
@@ -39,6 +39,7 @@
#include <linux/buffer_head.h> /* for invalidate_bdev */
#include <linux/poll.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/hdreg.h>
#include <linux/proc_fs.h>
#include <linux/random.h>
@@ -3246,8 +3247,7 @@ bitmap_store(mddev_t *mddev, const char
}
if (*end && !isspace(*end)) break;
bitmap_dirty_bits(mddev->bitmap, chunk, end_chunk);
- buf = end;
- while (isspace(*buf)) buf++;
+ buf = skip_spaces(end);
}
bitmap_unplug(mddev->bitmap); /* flush the bits to disk */
out:
diff -puN drivers/parisc/pdc_stable.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/parisc/pdc_stable.c
--- a/drivers/parisc/pdc_stable.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/parisc/pdc_stable.c
@@ -779,12 +779,9 @@ static ssize_t pdcs_auto_write(struct ko
read_unlock(&pathentry->rw_lock);

DPRINTK("%s: flags before: 0x%X\n", __func__, flags);
-
- temp = in;
-
- while (*temp && isspace(*temp))
- temp++;
-
+
+ temp = skip_spaces(in);
+
c = *temp++ - '0';
if ((c != 0) && (c != 1))
goto parse_error;
diff -puN drivers/platform/x86/thinkpad_acpi.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/platform/x86/thinkpad_acpi.c
--- a/drivers/platform/x86/thinkpad_acpi.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/platform/x86/thinkpad_acpi.c
@@ -1006,11 +1006,8 @@ static int parse_strtoul(const char *buf
{
char *endp;

- while (*buf && isspace(*buf))
- buf++;
- *value = simple_strtoul(buf, &endp, 0);
- while (*endp && isspace(*endp))
- endp++;
+ *value = simple_strtoul(skip_spaces(buf), &endp, 0);
+ endp = skip_spaces(endp);
if (*endp || *value > max)
return -EINVAL;

diff -puN drivers/pnp/interface.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/pnp/interface.c
--- a/drivers/pnp/interface.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/pnp/interface.c
@@ -310,8 +310,7 @@ static ssize_t pnp_set_current_resources
goto done;
}

- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (!strnicmp(buf, "disable", 7)) {
retval = pnp_disable_dev(dev);
goto done;
@@ -353,19 +352,13 @@ static ssize_t pnp_set_current_resources
pnp_init_resources(dev);
mutex_lock(&pnp_res_mutex);
while (1) {
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (!strnicmp(buf, "io", 2)) {
- buf += 2;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 2);
start = simple_strtoul(buf, &buf, 0);
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (*buf == '-') {
- buf += 1;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 1);
end = simple_strtoul(buf, &buf, 0);
} else
end = start;
@@ -373,16 +366,11 @@ static ssize_t pnp_set_current_resources
continue;
}
if (!strnicmp(buf, "mem", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf);
if (*buf == '-') {
- buf += 1;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 1);
end = simple_strtoul(buf, &buf, 0);
} else
end = start;
@@ -390,17 +378,13 @@ static ssize_t pnp_set_current_resources
continue;
}
if (!strnicmp(buf, "irq", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
pnp_add_irq_resource(dev, start, 0);
continue;
}
if (!strnicmp(buf, "dma", 3)) {
- buf += 3;
- while (isspace(*buf))
- ++buf;
+ buf = skip_spaces(buf + 3);
start = simple_strtoul(buf, &buf, 0);
pnp_add_dma_resource(dev, start, 0);
continue;
diff -puN drivers/s390/block/dasd_proc.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/s390/block/dasd_proc.c
--- a/drivers/s390/block/dasd_proc.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/s390/block/dasd_proc.c
@@ -14,6 +14,7 @@
#define KMSG_COMPONENT "dasd"

#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/seq_file.h>
#include <linux/vmalloc.h>
#include <linux/proc_fs.h>
@@ -272,10 +273,10 @@ dasd_statistics_write(struct file *file,
DBF_EVENT(DBF_DEBUG, "/proc/dasd/statictics: '%s'\n", buffer);

/* check for valid verbs */
- for (str = buffer; isspace(*str); str++);
+ str = skip_spaces(buffer);
if (strncmp(str, "set", 3) == 0 && isspace(str[3])) {
/* 'set xxx' was given */
- for (str = str + 4; isspace(*str); str++);
+ str = skip_spaces(str + 4);
if (strcmp(str, "on") == 0) {
/* switch on statistics profiling */
dasd_profile_level = DASD_PROFILE_ON;
diff -puN drivers/video/backlight/lcd.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/backlight/lcd.c
--- a/drivers/video/backlight/lcd.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/backlight/lcd.c
@@ -101,7 +101,7 @@ static ssize_t lcd_store_power(struct de
int power = simple_strtoul(buf, &endp, 0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
@@ -140,7 +140,7 @@ static ssize_t lcd_store_contrast(struct
int contrast = simple_strtoul(buf, &endp, 0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
diff -puN drivers/video/display/display-sysfs.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/display/display-sysfs.c
--- a/drivers/video/display/display-sysfs.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/display/display-sysfs.c
@@ -67,7 +67,7 @@ static ssize_t display_store_contrast(st
contrast = simple_strtoul(buf, &endp, 0);
size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;

if (size != count)
diff -puN drivers/video/output.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function drivers/video/output.c
--- a/drivers/video/output.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/drivers/video/output.c
@@ -50,7 +50,7 @@ static ssize_t video_output_store_state(
int request_state = simple_strtoul(buf,&endp,0);
size_t size = endp - buf;

- if (*endp && isspace(*endp))
+ if (isspace(*endp))
size++;
if (size != count)
return -EINVAL;
diff -puN fs/cachefiles/daemon.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function fs/cachefiles/daemon.c
--- a/fs/cachefiles/daemon.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/fs/cachefiles/daemon.c
@@ -21,6 +21,7 @@
#include <linux/mount.h>
#include <linux/statfs.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/fs_struct.h>
#include "internal.h"

@@ -257,8 +258,7 @@ static ssize_t cachefiles_daemon_write(s
if (args == data)
goto error;
*args = '\0';
- for (args++; isspace(*args); args++)
- continue;
+ args = skip_spaces(++args);
}

/* run the appropriate command handler */
diff -puN fs/ext4/super.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function fs/ext4/super.c
--- a/fs/ext4/super.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/fs/ext4/super.c
@@ -2137,11 +2137,8 @@ static int parse_strtoul(const char *buf
{
char *endp;

- while (*buf && isspace(*buf))
- buf++;
- *value = simple_strtoul(buf, &endp, 0);
- while (*endp && isspace(*endp))
- endp++;
+ *value = simple_strtoul(skip_spaces(buf), &endp, 0);
+ endp = skip_spaces(endp);
if (*endp || *value > max)
return -EINVAL;

diff -puN kernel/params.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function kernel/params.c
--- a/kernel/params.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/kernel/params.c
@@ -24,6 +24,7 @@
#include <linux/err.h>
#include <linux/slab.h>
#include <linux/ctype.h>
+#include <linux/string.h>

#if 0
#define DEBUGP printk
@@ -122,9 +123,7 @@ static char *next_arg(char *args, char *
next = args + i;

/* Chew up trailing spaces. */
- while (isspace(*next))
- next++;
- return next;
+ return skip_spaces(next);
}

/* Args looks like "foo=bar,bar2 baz=fuz wiz". */
@@ -139,8 +138,7 @@ int parse_args(const char *name,
DEBUGP("Parsing ARGS: %s\n", args);

/* Chew leading spaces */
- while (isspace(*args))
- args++;
+ args = skip_spaces(args);

while (*args) {
int ret;
diff -puN lib/argv_split.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/argv_split.c
--- a/lib/argv_split.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/argv_split.c
@@ -4,17 +4,10 @@

#include <linux/kernel.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/slab.h>
#include <linux/module.h>

-static const char *skip_sep(const char *cp)
-{
- while (*cp && isspace(*cp))
- cp++;
-
- return cp;
-}
-
static const char *skip_arg(const char *cp)
{
while (*cp && !isspace(*cp))
@@ -28,7 +21,7 @@ static int count_argc(const char *str)
int count = 0;

while (*str) {
- str = skip_sep(str);
+ str = skip_spaces(str);
if (*str) {
count++;
str = skip_arg(str);
@@ -82,7 +75,7 @@ char **argv_split(gfp_t gfp, const char
argvp = argv;

while (*str) {
- str = skip_sep(str);
+ str = skip_spaces(str);

if (*str) {
const char *p = str;
diff -puN lib/dynamic_debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/dynamic_debug.c
--- a/lib/dynamic_debug.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/dynamic_debug.c
@@ -21,6 +21,7 @@
#include <linux/list.h>
#include <linux/sysctl.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/dynamic_debug.h>
#include <linux/debugfs.h>
@@ -209,8 +210,7 @@ static int ddebug_tokenize(char *buf, ch
char *end;

/* Skip leading whitespace */
- while (*buf && isspace(*buf))
- buf++;
+ buf = skip_spaces(buf);
if (!*buf)
break; /* oh, it was trailing whitespace */

diff -puN lib/vsprintf.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function lib/vsprintf.c
--- a/lib/vsprintf.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/lib/vsprintf.c
@@ -1766,13 +1766,6 @@ EXPORT_SYMBOL_GPL(bprintf);

#endif /* CONFIG_BINARY_PRINTF */

-static noinline char *skip_space(const char *str)
-{
- while (isspace(*str))
- ++str;
- return (char *)str;
-}
-
/**
* vsscanf - Unformat a buffer into a list of arguments
* @buf: input buffer
@@ -1794,8 +1787,8 @@ int vsscanf(const char *buf, const char
* white space, including none, in the input.
*/
if (isspace(*fmt)) {
- fmt = skip_space(fmt);
- str = skip_space(str);
+ fmt = skip_spaces(++fmt);
+ str = skip_spaces(str);
}

/* anything that is not a conversion must match exactly */
@@ -1865,7 +1858,7 @@ int vsscanf(const char *buf, const char
if (field_width == -1)
field_width = INT_MAX;
/* first, skip leading white space in buffer */
- str = skip_space(str);
+ str = skip_spaces(str);

/* now copy until next white space */
while (*str && !isspace(*str) && field_width--)
@@ -1907,7 +1900,7 @@ int vsscanf(const char *buf, const char
/* have some sort of integer conversion.
* first, skip white space in buffer.
*/
- str = skip_space(str);
+ str = skip_spaces(str);

digit = *str;
if (is_sign && digit == '-')
diff -puN net/irda/irnet/irnet.h~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/irda/irnet/irnet.h
--- a/net/irda/irnet/irnet.h~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/irda/irnet/irnet.h
@@ -249,6 +249,7 @@
#include <linux/poll.h>
#include <linux/capability.h>
#include <linux/ctype.h> /* isspace() */
+#include <linux/string.h> /* skip_spaces() */
#include <asm/uaccess.h>
#include <linux/init.h>

diff -puN net/irda/irnet/irnet_ppp.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/irda/irnet/irnet_ppp.c
--- a/net/irda/irnet/irnet_ppp.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/irda/irnet/irnet_ppp.c
@@ -76,9 +76,8 @@ irnet_ctrl_write(irnet_socket * ap,
/* Look at the next command */
start = next;

- /* Scrap whitespaces before the command */
- while(isspace(*start))
- start++;
+ /* Scrap whitespaces before the command */
+ start = skip_spaces(start);

/* ',' is our command separator */
next = strchr(start, ',');
@@ -133,8 +132,7 @@ irnet_ctrl_write(irnet_socket * ap,
char * endp;

/* Scrap whitespaces before the command */
- while(isspace(*begp))
- begp++;
+ begp = skip_spaces(begp);

/* Convert argument to a number (last arg is the base) */
addr = simple_strtoul(begp, &endp, 16);
diff -puN net/netfilter/xt_recent.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function net/netfilter/xt_recent.c
--- a/net/netfilter/xt_recent.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/net/netfilter/xt_recent.c
@@ -482,8 +482,7 @@ static ssize_t recent_old_proc_write(str
if (copy_from_user(buf, input, size))
return -EFAULT;

- while (isspace(*c))
- c++;
+ c = skip_spaces(c);

if (size - (c - buf) < 5)
return c - buf;
diff -puN sound/pci/hda/hda_hwdep.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function sound/pci/hda/hda_hwdep.c
--- a/sound/pci/hda/hda_hwdep.c~tree-wide-convert-open-calls-to-remove-spaces-to-skip_spaces-lib-function
+++ a/sound/pci/hda/hda_hwdep.c
@@ -24,6 +24,7 @@
#include <linux/compat.h>
#include <linux/mutex.h>
#include <linux/ctype.h>
+#include <linux/string.h>
#include <linux/firmware.h>
#include <sound/core.h>
#include "hda_codec.h"
@@ -428,8 +429,7 @@ static int parse_hints(struct hda_codec
char *key, *val;
struct hda_hint *hint;

- while (isspace(*buf))
- buf++;
+ buf = skip_spaces(buf);
if (!*buf || *buf == '#' || *buf == '\n')
return 0;
if (*buf == '=')
@@ -444,8 +444,7 @@ static int parse_hints(struct hda_codec
return -EINVAL;
}
*val++ = 0;
- while (isspace(*val))
- val++;
+ val = skip_spaces(val);
remove_trail_spaces(key);
remove_trail_spaces(val);
hint = get_hint(codec, key);
_
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH] ext3: Fix data / filesystem corruption when write fails to copy data

2009-12-14T12:35:57Z

When ext3_write_begin fails after allocating some blocks or
generic_perform_write fails to copy data to write, we truncate blocks already
instantiated beyond i_size. Although these blocks were never inside i_size, we
have to truncate pagecache of these blocks so that corresponding buffers get
unmapped. Otherwise subsequent __block_prepare_write (called because we are
retrying the write) will find the buffers mapped, not call ->get_block, and
thus the page will be backed by already freed blocks leading to filesystem and
data corruption.

Reported-by: James Y Knight <foom@...>
Signed-off-by: Jan Kara <jack@...>
---
fs/ext3/inode.c | 18 ++++++++++++++----
1 files changed, 14 insertions(+), 4 deletions(-)

The bug has been introduced by commit 9eaaa2d5759837402ec5eee13b2a97921808c3eb
and thus this fix needs to be applied to 2.6.31 and 2.6.32 kernels. Currently,
it's in Linus's tree as commit 68eb3db08344286733adac48304d9fb7a0e53b27.

diff --git a/fs/ext3/inode.c b/fs/ext3/inode.c
index 354ed3b..f9d6937 100644
--- a/fs/ext3/inode.c
+++ b/fs/ext3/inode.c
@@ -1151,6 +1151,16 @@ static int do_journal_get_write_access(handle_t *handle,
return ext3_journal_get_write_access(handle, bh);
}

+/*
+ * Truncate blocks that were not used by write. We have to truncate the
+ * pagecache as well so that corresponding buffers get properly unmapped.
+ */
+static void ext3_truncate_failed_write(struct inode *inode)
+{
+ truncate_inode_pages(inode->i_mapping, inode->i_size);
+ ext3_truncate(inode);
+}
+
static int ext3_write_begin(struct file *file, struct address_space *mapping,
loff_t pos, unsigned len, unsigned flags,
struct page **pagep, void **fsdata)
@@ -1209,7 +1219,7 @@ write_begin_failed:
unlock_page(page);
page_cache_release(page);
if (pos + len > inode->i_size)
- ext3_truncate(inode);
+ ext3_truncate_failed_write(inode);
}
if (ret == -ENOSPC && ext3_should_retry_alloc(inode->i_sb, &retries))
goto retry;
@@ -1304,7 +1314,7 @@ static int ext3_ordered_write_end(struct file *file,
page_cache_release(page);

if (pos + len > inode->i_size)
- ext3_truncate(inode);
+ ext3_truncate_failed_write(inode);
return ret ? ret : copied;
}

@@ -1330,7 +1340,7 @@ static int ext3_writeback_write_end(struct file *file,
page_cache_release(page);

if (pos + len > inode->i_size)
- ext3_truncate(inode);
+ ext3_truncate_failed_write(inode);
return ret ? ret : copied;
}

@@ -1383,7 +1393,7 @@ static int ext3_journalled_write_end(struct file *file,
page_cache_release(page);

if (pos + len > inode->i_size)
- ext3_truncate(inode);
+ ext3_truncate_failed_write(inode);
return ret ? ret : copied;
}

--
1.6.4.2

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] ext3: Replace lock/unlock_super() with an explicit lock for resizing

2009-12-14T11:01:05Z

From: Theodore Ts'o <tytso@...>

Use a separate lock to protect s_groups_count and the other block
group descriptors which get changed via an on-line resize operation,
so we can stop overloading the use of lock_super().

Crossport of 32ed5058ce90024efcd811254b4b1de0468099df

Signed-off-by: Eric Sandeen <sandeen@...>
---
fs/ext3/resize.c | 35 ++++++++++++++++++-----------------
fs/ext3/super.c | 1 +
include/linux/ext3_fs_sb.h | 1 +
3 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/fs/ext3/resize.c b/fs/ext3/resize.c
index 8359e7b..57a5bc9 100644
--- a/fs/ext3/resize.c
+++ b/fs/ext3/resize.c
@@ -209,7 +209,7 @@ static int setup_new_group_blocks(struct super_block *sb,
if (IS_ERR(handle))
return PTR_ERR(handle);

- lock_super(sb);
+ mutex_lock(&sbi->s_resize_lock);
if (input->group != sbi->s_groups_count) {
err = -EBUSY;
goto exit_journal;
@@ -324,7 +324,7 @@ exit_bh:
brelse(bh);

exit_journal:
- unlock_super(sb);
+ mutex_unlock(&sbi->s_resize_lock);
if ((err2 = ext3_journal_stop(handle)) && !err)
err = err2;

@@ -662,11 +662,12 @@ exit_free:
* important part is that the new block and inode counts are in the backup
* superblocks, and the location of the new group metadata in the GDT backups.
*
- * We do not need lock_super() for this, because these blocks are not
- * otherwise touched by the filesystem code when it is mounted. We don't
- * need to worry about last changing from sbi->s_groups_count, because the
- * worst that can happen is that we do not copy the full number of backups
- * at this time. The resize which changed s_groups_count will backup again.
+ * We do not need take the s_resize_lock for this, because these
+ * blocks are not otherwise touched by the filesystem code when it is
+ * mounted. We don't need to worry about last changing from
+ * sbi->s_groups_count, because the worst that can happen is that we
+ * do not copy the full number of backups at this time. The resize
+ * which changed s_groups_count will backup again.
*/
static void update_backups(struct super_block *sb,
int blk_off, char *data, int size)
@@ -825,7 +826,7 @@ int ext3_group_add(struct super_block *sb, struct ext3_new_group_data *input)
goto exit_put;
}

- lock_super(sb);
+ mutex_lock(&sbi->s_resize_lock);
if (input->group != sbi->s_groups_count) {
ext3_warning(sb, __func__,
"multiple resizers run on filesystem!");
@@ -856,7 +857,7 @@ int ext3_group_add(struct super_block *sb, struct ext3_new_group_data *input)
/*
* OK, now we've set up the new group. Time to make it active.
*
- * Current kernels don't lock all allocations via lock_super(),
+ * We do not lock all allocations via s_resize_lock
* so we have to be safe wrt. concurrent accesses the group
* data. So we need to be careful to set all of the relevant
* group descriptor data etc. *before* we enable the group.
@@ -900,12 +901,12 @@ int ext3_group_add(struct super_block *sb, struct ext3_new_group_data *input)
*
* The precise rules we use are:
*
- * * Writers of s_groups_count *must* hold lock_super
+ * * Writers of s_groups_count *must* hold s_resize_lock
* AND
* * Writers must perform a smp_wmb() after updating all dependent
* data and before modifying the groups count
*
- * * Readers must hold lock_super() over the access
+ * * Readers must hold s_resize_lock over the access
* OR
* * Readers must perform an smp_rmb() after reading the groups count
* and before reading any dependent data.
@@ -936,7 +937,7 @@ int ext3_group_add(struct super_block *sb, struct ext3_new_group_data *input)
ext3_journal_dirty_metadata(handle, sbi->s_sbh);

exit_journal:
- unlock_super(sb);
+ mutex_unlock(&sbi->s_resize_lock);
if ((err2 = ext3_journal_stop(handle)) && !err)
err = err2;
if (!err) {
@@ -973,7 +974,7 @@ int ext3_group_extend(struct super_block *sb, struct ext3_super_block *es,

/* We don't need to worry about locking wrt other resizers just
* yet: we're going to revalidate es->s_blocks_count after
- * taking lock_super() below. */
+ * taking the s_resize_lock below. */
o_blocks_count = le32_to_cpu(es->s_blocks_count);
o_groups_count = EXT3_SB(sb)->s_groups_count;

@@ -1045,11 +1046,11 @@ int ext3_group_extend(struct super_block *sb, struct ext3_super_block *es,
goto exit_put;
}

- lock_super(sb);
+ mutex_lock(&EXT3_SB(sb)->s_resize_lock);
if (o_blocks_count != le32_to_cpu(es->s_blocks_count)) {
ext3_warning(sb, __func__,
"multiple resizers run on filesystem!");
- unlock_super(sb);
+ mutex_unlock(&EXT3_SB(sb)->s_resize_lock);
ext3_journal_stop(handle);
err = -EBUSY;
goto exit_put;
@@ -1059,13 +1060,13 @@ int ext3_group_extend(struct super_block *sb, struct ext3_super_block *es,
EXT3_SB(sb)->s_sbh))) {
ext3_warning(sb, __func__,
"error %d on journal write access", err);
- unlock_super(sb);
+ mutex_unlock(&EXT3_SB(sb)->s_resize_lock);
ext3_journal_stop(handle);
goto exit_put;
}
es->s_blocks_count = cpu_to_le32(o_blocks_count + add);
ext3_journal_dirty_metadata(handle, EXT3_SB(sb)->s_sbh);
- unlock_super(sb);
+ mutex_unlock(&EXT3_SB(sb)->s_resize_lock);
ext3_debug("freeing blocks %lu through "E3FSBLK"\n", o_blocks_count,
o_blocks_count + add);
ext3_free_blocks_sb(handle, sb, o_blocks_count, add, &freed_blocks);
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 534557c..53d3c53 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -1891,6 +1891,7 @@ static int ext3_fill_super (struct super_block *sb, void *data, int silent)
#endif
INIT_LIST_HEAD(&sbi->s_orphan); /* unlinked but open files */
mutex_init(&sbi->s_orphan_lock);
+ mutex_init(&sbi->s_resize_lock);

sb->s_root = NULL;

diff --git a/include/linux/ext3_fs_sb.h b/include/linux/ext3_fs_sb.h
index dd61d83..258088a 100644
--- a/include/linux/ext3_fs_sb.h
+++ b/include/linux/ext3_fs_sb.h
@@ -73,6 +73,7 @@ struct ext3_sb_info {
struct journal_s * s_journal;
struct list_head s_orphan;
struct mutex s_orphan_lock;
+ struct mutex s_resize_lock;
unsigned long s_commit_interval;
struct block_device *journal_bdev;
#ifdef CONFIG_JBD_DEBUG
-- 1.6.0.6

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] ext3: Replace lock/unlock_super() with an explicit lock for the orphan list

2009-12-14T11:00:30Z

From: Theodore Ts'o <tytso@...>

Use a separate lock to protect the orphan list, so we can stop
overloading the use of lock_super().

Crossport of 3b9d4ed26680771295d904a6b83e88e620780893

Signed-off-by: Eric Sandeen <sandeen@...>
---
fs/ext3/namei.c | 20 +++++++++++---------
fs/ext3/super.c | 1 +
include/linux/ext3_fs_sb.h | 1 +
3 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/fs/ext3/namei.c b/fs/ext3/namei.c
index aad6400..76d4c58 100644
--- a/fs/ext3/namei.c
+++ b/fs/ext3/namei.c
@@ -1920,7 +1920,7 @@ int ext3_orphan_add(handle_t *handle, struct inode *inode)
struct ext3_iloc iloc;
int err = 0, rc;

- lock_super(sb);
+ mutex_lock(&EXT3_SB(sb)->s_orphan_lock);
if (!list_empty(&EXT3_I(inode)->i_orphan))
goto out_unlock;

@@ -1929,9 +1929,13 @@ int ext3_orphan_add(handle_t *handle, struct inode *inode)

/* @@@ FIXME: Observation from aviro:
* I think I can trigger J_ASSERT in ext3_orphan_add(). We block
- * here (on lock_super()), so race with ext3_link() which might bump
+ * here (on s_orphan_lock), so race with ext3_link() which might bump
* ->i_nlink. For, say it, character device. Not a regular file,
* not a directory, not a symlink and ->i_nlink > 0.
+ *
+ * tytso, 4/25/2009: I'm not sure how that could happen;
+ * shouldn't the fs core protect us from these sort of
+ * unlink()/link() races?
*/
J_ASSERT ((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) ||
S_ISLNK(inode->i_mode)) || inode->i_nlink == 0);
@@ -1968,7 +1972,7 @@ int ext3_orphan_add(handle_t *handle, struct inode *inode)
jbd_debug(4, "orphan inode %lu will point to %d\n",
inode->i_ino, NEXT_ORPHAN(inode));
out_unlock:
- unlock_super(sb);
+ mutex_unlock(&EXT3_SB(sb)->s_orphan_lock);
ext3_std_error(inode->i_sb, err);
return err;
}
@@ -1986,11 +1990,9 @@ int ext3_orphan_del(handle_t *handle, struct inode *inode)
struct ext3_iloc iloc;
int err = 0;

- lock_super(inode->i_sb);
- if (list_empty(&ei->i_orphan)) {
- unlock_super(inode->i_sb);
- return 0;
- }
+ mutex_lock(&EXT3_SB(inode->i_sb)->s_orphan_lock);
+ if (list_empty(&ei->i_orphan))
+ goto out;

ino_next = NEXT_ORPHAN(inode);
prev = ei->i_orphan.prev;
@@ -2040,7 +2042,7 @@ int ext3_orphan_del(handle_t *handle, struct inode *inode)
out_err:
ext3_std_error(inode->i_sb, err);
out:
- unlock_super(inode->i_sb);
+ mutex_unlock(&EXT3_SB(inode->i_sb)->s_orphan_lock);
return err;

out_brelse:
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 602b308..534557c 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -1890,6 +1890,7 @@ static int ext3_fill_super (struct super_block *sb, void *data, int silent)
sb->dq_op = &ext3_quota_operations;
#endif
INIT_LIST_HEAD(&sbi->s_orphan); /* unlinked but open files */
+ mutex_init(&sbi->s_orphan_lock);

sb->s_root = NULL;

diff --git a/include/linux/ext3_fs_sb.h b/include/linux/ext3_fs_sb.h
index f07f34d..dd61d83 100644
--- a/include/linux/ext3_fs_sb.h
+++ b/include/linux/ext3_fs_sb.h
@@ -72,6 +72,7 @@ struct ext3_sb_info {
struct inode * s_journal_inode;
struct journal_s * s_journal;
struct list_head s_orphan;
+ struct mutex s_orphan_lock;
unsigned long s_commit_interval;
struct block_device *journal_bdev;
#ifdef CONFIG_JBD_DEBUG
-- 1.6.0.6

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 2/4] ext3: ext3_mark_recovery_complete() doesn't need to use lock_super

2009-12-14T10:59:59Z

From: Theodore Ts'o <tytso@...>

The function ext3_mark_recovery_complete() is called from two call
paths: either (a) while mounting the filesystem, in which case there's
no danger of any other CPU calling write_super() until the mount is
completed, and (b) while remounting the filesystem read-write, in
which case the fs core has already locked the superblock. This also
allows us to take out a very vile unlock_super()/lock_super() pair in
ext3_remount().

Crossport of a63c9eb2ce6f5028da90f282798232c4f398ceb8

Signed-off-by: Eric Sandeen <sandeen@...>
---
fs/ext3/super.c | 8 --------
1 files changed, 0 insertions(+), 8 deletions(-)

diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index bed624c..602b308 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -2352,13 +2352,11 @@ static void ext3_mark_recovery_complete(struct super_block * sb,
if (journal_flush(journal) < 0)
goto out;

- lock_super(sb);
if (EXT3_HAS_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER) &&
sb->s_flags & MS_RDONLY) {
EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
ext3_commit_super(sb, es, 1);
}
- unlock_super(sb);

out:
journal_unlock_updates(journal);
@@ -2550,13 +2548,7 @@ static int ext3_remount (struct super_block * sb, int * flags, char * data)
(sbi->s_mount_state & EXT3_VALID_FS))
es->s_state = cpu_to_le16(sbi->s_mount_state);

- /*
- * We have to unlock super so that we can wait for
- * transactions.
- */
- unlock_super(sb);
ext3_mark_recovery_complete(sb, es);
- lock_super(sb);
} else {
__le32 ret;
if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb,
-- 1.6.0.6

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html

[PATCH 1/4] ext3: Remove outdated comment about lock_super()

2009-12-14T10:59:18Z

From: Theodore Ts'o <tytso@...>

ext3_fill_super() is no longer called by read_super(), and it is no
longer called with the superblock locked. The
unlock_super()/lock_super() is no longer present, so this comment is
entirely superfluous.

Crossport of 32ed5058ce90024efcd811254b4b1de0468099df

Signed-off-by: Eric Sandeen <sandeen@...>
---
fs/ext3/super.c | 9 +--------
1 files changed, 1 insertions(+), 8 deletions(-)

diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 427496c..bed624c 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -1974,14 +1974,7 @@ static int ext3_fill_super (struct super_block *sb, void *data, int silent)
}

ext3_setup_super (sb, es, sb->s_flags & MS_RDONLY);
- /*
- * akpm: core read_super() calls in here with the superblock locked.
- * That deadlocks, because orphan cleanup needs to lock the superblock
- * in numerous places. Here we just pop the lock - it's relatively
- * harmless, because we are now ready to accept write_super() requests,
- * and aviro says that's the only reason for hanging onto the
- * superblock lock.
- */
+
EXT3_SB(sb)->s_mount_state |= EXT3_ORPHAN_FS;
ext3_orphan_cleanup(sb, es);
EXT3_SB(sb)->s_mount_state &= ~EXT3_ORPHAN_FS;
-- 1.6.0.6

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...
More majordomo info at http://vger.kernel.org/majordomo-info.html