linux.git/block/bio.c, branch v4.14.145

block: bio_iov_iter_get_pages: pin more pages for multi-segment IOs

2019-07-03T11:15:58+00:00

[ Upstream commit 17d51b10d7773e4618bcac64648f30f12d4078fb ]

bio_iov_iter_get_pages() currently only adds pages for the next non-zero
segment from the iov_iter to the bio. That's suboptimal for callers,
which typically try to pin as many pages as fit into the bio. This patch
converts the current bio_iov_iter_get_pages() into a static helper, and
introduces a new helper that allocates as many pages as

 1) fit into the bio,
 2) are present in the iov_iter,
 3) and can be pinned by MM.

Error is returned only if zero pages could be pinned. Because of 3), a
zero return value doesn't necessarily mean all pages have been pinned.
Callers that have to pin every page in the iov_iter must still call this
function in a loop (this is currently the case).

This change matters most for __blkdev_direct_IO_simple(), which calls
bio_iov_iter_get_pages() only once. If it obtains less pages than
requested, it returns a "short write" or "short read", and
__generic_file_write_iter() falls back to buffered writes, which may
lead to data corruption.

Fixes: 72ecad22d9f1 ("block: support a full bio worth of IO for simplified bdev direct-io")
Reviewed-by: Christoph Hellwig 
Signed-off-by: Martin Wilck 
Signed-off-by: Jens Axboe 
Signed-off-by: Sasha Levin

block: add a lower-level bio_add_page interface

2019-07-03T11:15:58+00:00

[ Upstream commit 0aa69fd32a5f766e997ca8ab4723c5a1146efa8b ]

For the upcoming removal of buffer heads in XFS we need to keep track of
the number of outstanding writeback requests per page.  For this we need
to know if bio_add_page merged a region with the previous bvec or not.
Instead of adding additional arguments this refactors bio_add_page to
be implemented using three lower level helpers which users like XFS can
use directly if they care about the merge decisions.

Signed-off-by: Christoph Hellwig 
Reviewed-by: Jens Axboe 
Reviewed-by: Ming Lei 
Reviewed-by: Darrick J. Wong 
Signed-off-by: Darrick J. Wong 
Signed-off-by: Sasha Levin

block: do not leak memory in bio_copy_user_iov()

2019-04-17T06:37:53+00:00

commit a3761c3c91209b58b6f33bf69dd8bb8ec0c9d925 upstream.

When bio_add_pc_page() fails in bio_copy_user_iov() we should free
the page we just allocated otherwise we are leaking it.

Cc: linux-block@vger.kernel.org
Cc: Linus Torvalds 
Cc: stable@vger.kernel.org
Reviewed-by: Chaitanya Kulkarni 
Signed-off-by: Jérôme Glisse 
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman

block: bvec_nr_vecs() returns value for wrong slab

2018-09-15T07:45:30+00:00

[ Upstream commit d6c02a9beb67f13d5f14f23e72fa9981e8b84477 ]

In commit ed996a52c868 ("block: simplify and cleanup bvec pool
handling"), the value of the slab index is incremented by one in
bvec_alloc() after the allocation is done to indicate an index value of
0 does not need to be later freed.

bvec_nr_vecs() was not updated accordingly, and thus returns the wrong
value.  Decrement idx before performing the lookup.

Fixes: ed996a52c868 ("block: simplify and cleanup bvec pool handling")
Signed-off-by: Greg Edwards 
Signed-off-by: Jens Axboe 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

block: reset bi_iter.bi_done after splitting bio

2018-08-03T05:50:42+00:00

commit 5151842b9d8732d4cbfa8400b40bff894f501b2f upstream.

After the bio has been updated to represent the remaining sectors, reset
bi_done so bio_rewind_iter() does not rewind further than it should.

This resolves a bio_integrity_process() failure on reads where the
original request was split.

Fixes: 63573e359d05 ("bio-integrity: Restore original iterator on verify stage")
Signed-off-by: Greg Edwards 
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman

block: bio_iov_iter_get_pages: fix size of last iovec

2018-08-03T05:50:42+00:00

commit b403ea2404889e1227812fa9657667a1deb9c694 upstream.

If the last page of the bio is not "full", the length of the last
vector slot needs to be corrected. This slot has the index
(bio->bi_vcnt - 1), but only in bio->bi_io_vec. In the "bv" helper
array, which is shifted by the value of bio->bi_vcnt at function
invocation, the correct index is (nr_pages - 1).

v2: improved readability following suggestions from Ming Lei.
v3: followed a formatting suggestion from Christoph Hellwig.

Fixes: 2cefe4dbaadf ("block: add bio_iov_iter_get_pages()")
Reviewed-by: Hannes Reinecke 
Reviewed-by: Ming Lei 
Reviewed-by: Jan Kara 
Reviewed-by: Christoph Hellwig 
Signed-off-by: Martin Wilck 
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman

block: Set BIO_TRACE_COMPLETION on new bio during split

2018-04-26T09:02:10+00:00

[ Upstream commit 20d59023c5ec4426284af492808bcea1f39787ef ]

We inadvertently set it again on the source bio, but we need
to set it on the new split bio instead.

Fixes: fbbaf700e7b1 ("block: trace completion of all bios.")
Signed-off-by: Goldwyn Rodrigues 
Signed-off-by: Jens Axboe 
Signed-off-by: Sasha Levin 
Signed-off-by: Greg Kroah-Hartman

Fix slab name "biovec-(1<<(21-12))"

2018-04-08T12:26:33+00:00

commit bd5c4facf59648581d2f1692dad7b107bf429954 upstream.

I'm getting a slab named "biovec-(1<<(21-12))". It is caused by unintended
expansion of the macro BIO_MAX_PAGES. This patch renames it to biovec-max.

Signed-off-by: Mikulas Patocka 
Cc: stable@vger.kernel.org	# v4.14+
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman

block-throttle: avoid double charge

2017-12-29T16:53:47+00:00

commit 111be883981748acc9a56e855c8336404a8e787c upstream.

If a bio is throttled and split after throttling, the bio could be
resubmited and enters the throttling again. This will cause part of the
bio to be charged multiple times. If the cgroup has an IO limit, the
double charge will significantly harm the performance. The bio split
becomes quite common after arbitrary bio size change.

To fix this, we always set the BIO_THROTTLED flag if a bio is throttled.
If the bio is cloned/split, we copy the flag to new bio too to avoid a
double charge. However, cloned bio could be directed to a new disk,
keeping the flag be a problem. The observation is we always set new disk
for the bio in this case, so we can clear the flag in bio_set_dev().

This issue exists for a long time, arbitrary bio size change just makes
it worse, so this should go into stable at least since v4.2.

V1-> V2: Not add extra field in bio based on discussion with Tejun

Cc: Vivek Goyal 
Acked-by: Tejun Heo 
Signed-off-by: Shaohua Li 
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman

bio: ensure __bio_clone_fast copies bi_partno

2017-11-24T07:37:03+00:00

commit 62530ed8b1d07a45dec94d46e521c0c6c2d476e6 upstream.

A new field was introduced in 74d46992e0d9, bi_partno, instead of using
bdev->bd_contains and encoding the partition information in the bi_bdev
field.  __bio_clone_fast was changed to copy the disk information, but
not the partition information.  At minimum, this regressed bcache and
caused data corruption.

Signed-off-by: Michael Lyle 
Fixes: 74d46992e0d9 ("block: replace bi_bdev with a gendisk pointer and partitions index")
Reported-by: Pavel Goran 
Reported-by: Campbell Steven 
Reviewed-by: Coly Li 
Reviewed-by: Ming Lei 
Signed-off-by: Jens Axboe 
Signed-off-by: Greg Kroah-Hartman