Write performance issue with cfi_cmdset_0001.c

[Josh Boyer]

On Fri, 2005-10-28 at 18:04 -0400, Nicolas Pitre wrote:
> 
> But current code has almost same behavior:

> And therefore I wonder if what you saw could occur with that version as 
> well.

Ok, here is a pseduo-description of the problem.  Sorry for the delay in
my reply, I've been trying to verify that it still exists in 2.6.  The
answer is "I think so" ;).

The basic issue is a starvation/contention issue on the flash chip
itself.  Assume you have 2 process.  Process A is writing a large chunk
of data to flash.  Process B is doing a read, erase, or some other
operation to the same chip.

In do_write_buffer, A puts the chip in FL_WRITING and then calls
cfi_udelay.  Assume A is scheduled out at this point and B is scheduled
in.  Process B attempts to get the chip to perform it's operation and
finds it can't because the state is FL_WRITING.  It is put on the chip
wait queue and scheduled out.

Process A resumes and finishes it's write.  At the end of
do_write_buffer, it places the chip state to FL_STATUS and calls
put_chip which calls wake_up on the chip's wait queue.  However, wake_up
does not actually call schedule, it just puts process B on the run
queue.  Process A is still the running thread at this point.

The wake_up also goes and recalculates the priorities of both process A,
and process B.  Since process B hasn't made any progress, it's priority
is better than that of A and A is marked as needing rescheduling.

As process A is still running, do_write_buffer returns and is
immediately called again by cfi_intelext_writev.  do_write_buffer then
puts the chip back in FL_WRITING can the same cycle repeats.

By adding a cond_resched after the do_write_buffer call in
cfi_intelext_writev, it allows the writing process to be scheduled out
and leave the chip in a state that is usable by other processes.

As I said, we did this in our kernel and it decreased write times quite
a bit in scenarios where we had such contention.  Namely when large
writes are being done (think 16MiB total in chunks of 256KiB).  In 2.6,
the problem is not quite so drastic as the sleep time of the writing
process is less due to a higher HZ value.  However, the general problem
does seem to exist.

Hope that is a bit more clear.  The problem itself is pretty icky, so
feel free to ask questions.

josh