1. 02 Oct, 2019 1 commit
    • Juergen Gross's avatar
      xen/xenbus: fix self-deadlock after killing user process · a8fabb38
      Juergen Gross authored
      
      
      In case a user process using xenbus has open transactions and is killed
      e.g. via ctrl-C the following cleanup of the allocated resources might
      result in a deadlock due to trying to end a transaction in the xenbus
      worker thread:
      
      [ 2551.474706] INFO: task xenbus:37 blocked for more than 120 seconds.
      [ 2551.492215]       Tainted: P           OE     5.0.0-29-generic #5
      [ 2551.510263] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      [ 2551.528585] xenbus          D    0    37      2 0x80000080
      [ 2551.528590] Call Trace:
      [ 2551.528603]  __schedule+0x2c0/0x870
      [ 2551.528606]  ? _cond_resched+0x19/0x40
      [ 2551.528632]  schedule+0x2c/0x70
      [ 2551.528637]  xs_talkv+0x1ec/0x2b0
      [ 2551.528642]  ? wait_woken+0x80/0x80
      [ 2551.528645]  xs_single+0x53/0x80
      [ 2551.528648]  xenbus_transaction_end+0x3b/0x70
      [ 2551.528651]  xenbus_file_free+0x5a/0x160
      [ 2551.528654]  xenbus_dev_queue_reply+0xc4/0x220
      [ 2551.528657]  xenbus_thread+0x7de/0x880
      [ 2551.528660]  ? wait_woken+0x80/0x80
      [ 2551.528665]  kthread+0x121/0x140
      [ 2551.528667]  ? xb_read+0x1d0/0x1d0
      [ 2551.528670]  ? kthread_park+0x90/0x90
      [ 2551.528673]  ret_from_fork+0x35/0x40
      
      Fix this by doing the cleanup via a workqueue instead.
      Reported-by: default avatarJames Dingwall <james@dingwall.me.uk>
      Fixes: fd8aa909
      
       ("xen: optimize xenbus driver for multiple concurrent xenstore accesses")
      Cc: <stable@vger.kernel.org> # 4.11
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Reviewed-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      Signed-off-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      a8fabb38
  2. 28 May, 2019 1 commit
    • Ross Lagerwall's avatar
      xenbus: Avoid deadlock during suspend due to open transactions · d10e0cc1
      Ross Lagerwall authored
      
      
      During a suspend/resume, the xenwatch thread waits for all outstanding
      xenstore requests and transactions to complete. This does not work
      correctly for transactions started by userspace because it waits for
      them to complete after freezing userspace threads which means the
      transactions have no way of completing, resulting in a deadlock. This is
      trivial to reproduce by running this script and then suspending the VM:
      
          import pyxs, time
          c = pyxs.client.Client(xen_bus_path="/dev/xen/xenbus")
          c.connect()
          c.transaction()
          time.sleep(3600)
      
      Even if this deadlock were resolved, misbehaving userspace should not
      prevent a VM from being migrated. So, instead of waiting for these
      transactions to complete before suspending, store the current generation
      id for each transaction when it is started. The global generation id is
      incremented during resume. If the caller commits the transaction and the
      generation id does not match the current generation id, return EAGAIN so
      that they try again. If the transaction was instead discarded, return OK
      since no changes were made anyway.
      
      This only affects users of the xenbus file interface. In-kernel users of
      xenbus are assumed to be well-behaved and complete all transactions
      before freezing.
      Signed-off-by: default avatarRoss Lagerwall <ross.lagerwall@citrix.com>
      Reviewed-by: default avatarJuergen Gross <jgross@suse.com>
      Signed-off-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      d10e0cc1
  3. 25 Apr, 2019 1 commit
  4. 06 Apr, 2019 1 commit
    • Kirill Smelkov's avatar
      fs: stream_open - opener for stream-like files so that read and write can run... · 10dce8af
      Kirill Smelkov authored
      fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock
      
      Commit 9c225f26 ("vfs: atomic f_pos accesses as per POSIX") added
      locking for file.f_pos access and in particular made concurrent read and
      write not possible - now both those functions take f_pos lock for the
      whole run, and so if e.g. a read is blocked waiting for data, write will
      deadlock waiting for that read to complete.
      
      This caused regression for stream-like files where previously read and
      write could run simultaneously, but after that patch could not do so
      anymore. See e.g. commit 581d21a2 ("xenbus: fix deadlock on writes
      to /proc/xen/xenbus") which fixes such regression for particular case of
      /proc/xen/xenbus.
      
      The patch that added f_pos lock in 2014 did so to guarantee POSIX thread
      safety for read/write/lseek and added the locking to file descriptors of
      all regular files. In 2014 that thread-safety problem was not new as it
      was already discussed earlier in 2006.
      
      However even though 2006'th version of Linus's patch was adding f_pos
      locking "only for files that are marked seekable with FMODE_LSEEK (thus
      avoiding the stream-like objects like pipes and sockets)", the 2014
      version - the one that actually made it into the tree as 9c225f26 -
      is doing so irregardless of whether a file is seekable or not.
      
      See
      
          https://lore.kernel.org/lkml/53022DB1.4070805@gmail.com/
          https://lwn.net/Articles/180387
          https://lwn.net/Articles/180396
      
      for historic context.
      
      The reason that it did so is, probably, that there are many files that
      are marked non-seekable, but e.g. their read implementation actually
      depends on knowing current position to correctly handle the read. Some
      examples:
      
      	kernel/power/user.c		snapshot_read
      	fs/debugfs/file.c		u32_array_read
      	fs/fuse/control.c		fuse_conn_waiting_read + ...
      	drivers/hwmon/asus_atk0110.c	atk_debugfs_ggrp_read
      	arch/s390/hypfs/inode.c		hypfs_read_iter
      	...
      
      Despite that, many nonseekable_open users implement read and write with
      pure stream semantics - they don't depend on passed ppos at all. And for
      those cases where read could wait for something inside, it creates a
      situation similar to xenbus - the write could be never made to go until
      read is done, and read is waiting for some, potentially external, event,
      for potentially unbounded time -> deadlock.
      
      Besides xenbus, there are 14 such places in the kernel that I've found
      with semantic patch (see below):
      
      	drivers/xen/evtchn.c:667:8-24: ERROR: evtchn_fops: .read() can deadlock .write()
      	drivers/isdn/capi/capi.c:963:8-24: ERROR: capi_fops: .read() can deadlock .write()
      	drivers/input/evdev.c:527:1-17: ERROR: evdev_fops: .read() can deadlock .write()
      	drivers/char/pcmcia/cm4000_cs.c:1685:7-23: ERROR: cm4000_fops: .read() can deadlock .write()
      	net/rfkill/core.c:1146:8-24: ERROR: rfkill_fops: .read() can deadlock .write()
      	drivers/s390/char/fs3270.c:488:1-17: ERROR: fs3270_fops: .read() can deadlock .write()
      	drivers/usb/misc/ldusb.c:310:1-17: ERROR: ld_usb_fops: .read() can deadlock .write()
      	drivers/hid/uhid.c:635:1-17: ERROR: uhid_fops: .read() can deadlock .write()
      	net/batman-adv/icmp_socket.c:80:1-17: ERROR: batadv_fops: .read() can deadlock .write()
      	drivers/media/rc/lirc_dev.c:198:1-17: ERROR: lirc_fops: .read() can deadlock .write()
      	drivers/leds/uleds.c:77:1-17: ERROR: uleds_fops: .read() can deadlock .write()
      	drivers/input/misc/uinput.c:400:1-17: ERROR: uinput_fops: .read() can deadlock .write()
      	drivers/infiniband/core/user_mad.c:985:7-23: ERROR: umad_fops: .read() can deadlock .write()
      	drivers/gnss/core.c:45:1-17: ERROR: gnss_fops: .read() can deadlock .write()
      
      In addition to the cases above another regression caused by f_pos
      locking is that now FUSE filesystems that implement open with
      FOPEN_NONSEEKABLE flag, can no longer implement bidirectional
      stream-like files - for the same reason as above e.g. read can deadlock
      write locking on file.f_pos in the kernel.
      
      FUSE's FOPEN_NONSEEKABLE was added in 2008 in a7c1b990 ("fuse:
      implement nonseekable open") to support OSSPD. OSSPD implements /dev/dsp
      in userspace with FOPEN_NONSEEKABLE flag, with corresponding read and
      write routines not depending on current position at all, and with both
      read and write being potentially blocking operations:
      
      See
      
          https://github.com/libfuse/osspd
          https://lwn.net/Articles/308445
      
          https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1406
          https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1438-L1477
          https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1479-L1510
      
      Corresponding libfuse example/test also describes FOPEN_NONSEEKABLE as
      "somewhat pipe-like files ..." with read handler not using offset.
      However that test implements only read without write and cannot exercise
      the deadlock scenario:
      
          https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L124-L131
          https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L146-L163
          https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L209-L216
      
      I've actually hit the read vs write deadlock for real while implementing
      my FUSE filesystem where there is /head/watch file, for which open
      creates separate bidirectional socket-like stream in between filesystem
      and its user with both read and write being later performed
      simultaneously. And there it is semantically not easy to split the
      stream into two separate read-only and write-only channels:
      
          https://lab.nexedi.com/kirr/wendelin.core/blob/f13aa600/wcfs/wcfs.go#L88-169
      
      Let's fix this regression. The plan is:
      
      1. We can't change nonseekable_open to include &~FMODE_ATOMIC_POS -
         doing so would break many in-kernel nonseekable_open users which
         actually use ppos in read/write handlers.
      
      2. Add stream_open() to kernel to open stream-like non-seekable file
         descriptors. Read and write on such file descriptors would never use
         nor change ppos. And with that property on stream-like files read and
         write will be running without taking f_pos lock - i.e. read and write
         could be running simultaneously.
      
      3. With semantic patch search and convert to stream_open all in-kernel
         nonseekable_open users for which read and write actually do not
         depend on ppos and where there is no other methods in file_operations
         which assume @offset access.
      
      4. Add FOPEN_STREAM to fs/fuse/ and open in-kernel file-descriptors via
         steam_open if that bit is present in filesystem open reply.
      
         It was tempting to change fs/fuse/ open handler to use stream_open
         instead of nonseekable_open on just FOPEN_NONSEEKABLE flags, but
         grepping through Debian codesearch shows users of FOPEN_NONSEEKABLE,
         and in particular GVFS which actually uses offset in its read and
         write handlers
      
      	https://codesearch.debian.net/search?q=-%3Enonseekable+%3D
      	https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1080
      	https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1247-1346
      	https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1399-1481
      
         so if we would do such a change it will break a real user.
      
      5. Add stream_open and FOPEN_STREAM handling to stable kernels starting
         from v3.14+ (the kernel where 9c225f26
      
       first appeared).
      
         This will allow to patch OSSPD and other FUSE filesystems that
         provide stream-like files to return FOPEN_STREAM | FOPEN_NONSEEKABLE
         in their open handler and this way avoid the deadlock on all kernel
         versions. This should work because fs/fuse/ ignores unknown open
         flags returned from a filesystem and so passing FOPEN_STREAM to a
         kernel that is not aware of this flag cannot hurt. In turn the kernel
         that is not aware of FOPEN_STREAM will be < v3.14 where just
         FOPEN_NONSEEKABLE is sufficient to implement streams without read vs
         write deadlock.
      
      This patch adds stream_open, converts /proc/xen/xenbus to it and adds
      semantic patch to automatically locate in-kernel places that are either
      required to be converted due to read vs write deadlock, or that are just
      safe to be converted because read and write do not use ppos and there
      are no other funky methods in file_operations.
      
      Regarding semantic patch I've verified each generated change manually -
      that it is correct to convert - and each other nonseekable_open instance
      left - that it is either not correct to convert there, or that it is not
      converted due to current stream_open.cocci limitations.
      
      The script also does not convert files that should be valid to convert,
      but that currently have .llseek = noop_llseek or generic_file_llseek for
      unknown reason despite file being opened with nonseekable_open (e.g.
      drivers/input/mousedev.c)
      
      Cc: Michael Kerrisk <mtk.manpages@gmail.com>
      Cc: Yongzhi Pan <panyongzhi@gmail.com>
      Cc: Jonathan Corbet <corbet@lwn.net>
      Cc: David Vrabel <david.vrabel@citrix.com>
      Cc: Juergen Gross <jgross@suse.com>
      Cc: Miklos Szeredi <miklos@szeredi.hu>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Christoph Hellwig <hch@lst.de>
      Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
      Cc: Julia Lawall <Julia.Lawall@lip6.fr>
      Cc: Nikolaus Rath <Nikolaus@rath.org>
      Cc: Han-Wen Nienhuys <hanwen@google.com>
      Signed-off-by: default avatarKirill Smelkov <kirr@nexedi.com>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      10dce8af
  5. 17 Apr, 2018 1 commit
  6. 21 Mar, 2018 2 commits
  7. 11 Feb, 2018 1 commit
    • Linus Torvalds's avatar
      vfs: do bulk POLL* -> EPOLL* replacement · a9a08845
      Linus Torvalds authored
      
      
      This is the mindless scripted replacement of kernel use of POLL*
      variables as described by Al, done by this script:
      
          for V in IN OUT PRI ERR RDNORM RDBAND WRNORM WRBAND HUP RDHUP NVAL MSG; do
              L=`git grep -l -w POLL$V | grep -v '^t' | grep -v /um/ | grep -v '^sa' | grep -v '/poll.h$'|grep -v '^D'`
              for f in $L; do sed -i "-es/^\([^\"]*\)\(\<POLL$V\>\)/\\1E\\2/" $f; done
          done
      
      with de-mangling cleanups yet to come.
      
      NOTE! On almost all architectures, the EPOLL* constants have the same
      values as the POLL* constants do.  But they keyword here is "almost".
      For various bad reasons they aren't the same, and epoll() doesn't
      actually work quite correctly in some cases due to this on Sparc et al.
      
      The next patch from Al will sort out the final differences, and we
      should be all done.
      Scripted-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      a9a08845
  8. 28 Nov, 2017 1 commit
  9. 04 Apr, 2017 1 commit
  10. 27 Feb, 2017 1 commit
  11. 09 Feb, 2017 3 commits
    • Juergen Gross's avatar
      xen: optimize xenbus driver for multiple concurrent xenstore accesses · fd8aa909
      Juergen Gross authored
      
      
      Handling of multiple concurrent Xenstore accesses through xenbus driver
      either from the kernel or user land is rather lame today: xenbus is
      capable to have one access active only at one point of time.
      
      Rewrite xenbus to handle multiple requests concurrently by making use
      of the request id of the Xenstore protocol. This requires to:
      
      - Instead of blocking inside xb_read() when trying to read data from
        the xenstore ring buffer do so only in the main loop of
        xenbus_thread().
      
      - Instead of doing writes to the xenstore ring buffer in the context of
        the caller just queue the request and do the write in the dedicated
        xenbus thread.
      
      - Instead of just forwarding the request id specified by the caller of
        xenbus to xenstore use a xenbus internal unique request id. This will
        allow multiple outstanding requests.
      
      - Modify the locking scheme in order to allow multiple requests being
        active in parallel.
      
      - Instead of waiting for the reply of a user's xenstore request after
        writing the request to the xenstore ring buffer return directly to
        the caller and do the waiting in the read path.
      
      Additionally signal handling was optimized by avoiding waking up the
      xenbus thread or sending an event to Xenstore in case the addressed
      entity is known to be running already.
      
      As a result communication with Xenstore is sped up by a factor of up
      to 5: depending on the request type (read or write) and the amount of
      data transferred the gain was at least 20% (small reads) and went up to
      a factor of 5 for large writes.
      
      In the end some more rough edges of xenbus have been smoothed:
      
      - Handling of memory shortage when reading from xenstore ring buffer in
        the xenbus driver was not optimal: it was busy looping and issuing a
        warning in each loop.
      
      - In case of xenstore not running in dom0 but in a stubdom we end up
        with two xenbus threads running as the initialization of xenbus in
        dom0 expecting a local xenstored will be redone later when connecting
        to the xenstore domain. Up to now this was no problem as locking
        would prevent the two xenbus threads interfering with each other, but
        this was just a waste of kernel resources.
      
      - An out of memory situation while writing to or reading from the
        xenstore ring buffer no longer will lead to a possible loss of
        synchronization with xenstore.
      
      - The user read and write part are now interruptible by signals.
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Signed-off-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      fd8aa909
    • Juergen Gross's avatar
      xen: modify xenstore watch event interface · 5584ea25
      Juergen Gross authored
      
      
      Today a Xenstore watch event is delivered via a callback function
      declared as:
      
      void (*callback)(struct xenbus_watch *,
                       const char **vec, unsigned int len);
      
      As all watch events only ever come with two parameters (path and token)
      changing the prototype to:
      
      void (*callback)(struct xenbus_watch *,
                       const char *path, const char *token);
      
      is the natural thing to do.
      
      Apply this change and adapt all users.
      
      Cc: konrad.wilk@oracle.com
      Cc: roger.pau@citrix.com
      Cc: wei.liu2@citrix.com
      Cc: paul.durrant@citrix.com
      Cc: netdev@vger.kernel.org
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Reviewed-by: default avatarPaul Durrant <paul.durrant@citrix.com>
      Reviewed-by: default avatarWei Liu <wei.liu2@citrix.com>
      Reviewed-by: default avatarRoger Pau Monné <roger.pau@citrix.com>
      Reviewed-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      Signed-off-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      5584ea25
    • Juergen Gross's avatar
      xen: clean up xenbus internal headers · 332f791d
      Juergen Gross authored
      
      
      The xenbus driver has an awful mixture of internally and globally
      visible headers: some of the internally used only stuff is defined in
      the global header include/xen/xenbus.h while some stuff defined in
      internal headers is used by other drivers, too.
      
      Clean this up by moving the externally used symbols to
      include/xen/xenbus.h and the symbols used internally only to a new
      header drivers/xen/xenbus/xenbus.h replacing xenbus_comms.h and
      xenbus_probe.h
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Reviewed-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      Signed-off-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      332f791d
  12. 23 Dec, 2016 2 commits
    • Juergen Gross's avatar
      xen: return xenstore command failures via response instead of rc · 9a6161fe
      Juergen Gross authored
      
      
      When the xenbus driver does some special handling for a Xenstore
      command any error condition related to the command should be returned
      via an error response instead of letting the related write operation
      fail. Otherwise the user land handler might take wrong decisions
      assuming the connection to Xenstore is broken.
      
      While at it try to return the same error values xenstored would
      return for those cases.
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Reviewed-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      9a6161fe
    • Juergen Gross's avatar
      xen: xenbus driver must not accept invalid transaction ids · 639b0881
      Juergen Gross authored
      
      
      When accessing Xenstore in a transaction the user is specifying a
      transaction id which he normally obtained from Xenstore when starting
      the transaction. Xenstore is validating a transaction id against all
      known transaction ids of the connection the request came in. As all
      requests of a domain not being the one where Xenstore lives share
      one connection, validation of transaction ids of different users of
      Xenstore in that domain should be done by the kernel of that domain
      being the multiplexer between the Xenstore users in that domain and
      Xenstore.
      
      In order to prohibit one Xenstore user "hijacking" a transaction from
      another user the xenbus driver has to verify a given transaction id
      against all known transaction ids of the user before forwarding it to
      Xenstore.
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      Reviewed-by: default avatarBoris Ostrovsky <boris.ostrovsky@oracle.com>
      Signed-off-by: default avatarJuergen Gross <jgross@suse.com>
      639b0881
  13. 12 Dec, 2016 1 commit
  14. 24 Oct, 2016 1 commit
  15. 24 Aug, 2016 1 commit
  16. 07 Jul, 2016 1 commit
  17. 21 Mar, 2016 1 commit
  18. 15 Feb, 2016 1 commit
  19. 05 Feb, 2015 1 commit
  20. 28 Jun, 2013 1 commit
    • Joe Perches's avatar
      xen: Convert printks to pr_<level> · 283c0972
      Joe Perches authored
      
      
      Convert printks to pr_<level> (excludes printk(KERN_DEBUG...)
      to be more consistent throughout the xen subsystem.
      
      Add pr_fmt with KBUILD_MODNAME or "xen:" KBUILD_MODNAME
      Coalesce formats and add missing word spaces
      Add missing newlines
      Align arguments and reflow to 80 columns
      Remove DRV_NAME from formats as pr_fmt adds the same content
      
      This does change some of the prefixes of these messages
      but it also does make them more consistent.
      Signed-off-by: default avatarJoe Perches <joe@perches.com>
      Signed-off-by: default avatarKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
      283c0972
  21. 30 Oct, 2012 1 commit
  22. 03 Feb, 2012 1 commit
  23. 04 Jan, 2012 1 commit
  24. 20 Dec, 2011 1 commit
  25. 16 Dec, 2011 2 commits
  26. 20 Dec, 2010 1 commit
  27. 15 Oct, 2010 1 commit
    • Arnd Bergmann's avatar
      llseek: automatically add .llseek fop · 6038f373
      Arnd Bergmann authored
      
      
      All file_operations should get a .llseek operation so we can make
      nonseekable_open the default for future file operations without a
      .llseek pointer.
      
      The three cases that we can automatically detect are no_llseek, seq_lseek
      and default_llseek. For cases where we can we can automatically prove that
      the file offset is always ignored, we use noop_llseek, which maintains
      the current behavior of not returning an error from a seek.
      
      New drivers should normally not use noop_llseek but instead use no_llseek
      and call nonseekable_open at open time.  Existing drivers can be converted
      to do the same when the maintainer knows for certain that no user code
      relies on calling seek on the device file.
      
      The generated code is often incorrectly indented and right now contains
      comments that clarify for each added line why a specific variant was
      chosen. In the version that gets submitted upstream, the comments will
      be gone and I will manually fix the indentation, because there does not
      seem to be a way to do that using coccinelle.
      
      Some amount of new code is currently sitting in linux-next that should get
      the same modifications, which I will do at the end of the merge window.
      
      Many thanks to Julia Lawall for helping me learn to write a semantic
      patch that does all this.
      
      ===== begin semantic patch =====
      // This adds an llseek= method to all file operations,
      // as a preparation for making no_llseek the default.
      //
      // The rules are
      // - use no_llseek explicitly if we do nonseekable_open
      // - use seq_lseek for sequential files
      // - use default_llseek if we know we access f_pos
      // - use noop_llseek if we know we don't access f_pos,
      //   but we still want to allow users to call lseek
      //
      @ open1 exists @
      identifier nested_open;
      @@
      nested_open(...)
      {
      <+...
      nonseekable_open(...)
      ...+>
      }
      
      @ open exists@
      identifier open_f;
      identifier i, f;
      identifier open1.nested_open;
      @@
      int open_f(struct inode *i, struct file *f)
      {
      <+...
      (
      nonseekable_open(...)
      |
      nested_open(...)
      )
      ...+>
      }
      
      @ read disable optional_qualifier exists @
      identifier read_f;
      identifier f, p, s, off;
      type ssize_t, size_t, loff_t;
      expression E;
      identifier func;
      @@
      ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
      {
      <+...
      (
         *off = E
      |
         *off += E
      |
         func(..., off, ...)
      |
         E = *off
      )
      ...+>
      }
      
      @ read_no_fpos disable optional_qualifier exists @
      identifier read_f;
      identifier f, p, s, off;
      type ssize_t, size_t, loff_t;
      @@
      ssize_t read_f(struct file *f, char *p, size_t s, loff_t *off)
      {
      ... when != off
      }
      
      @ write @
      identifier write_f;
      identifier f, p, s, off;
      type ssize_t, size_t, loff_t;
      expression E;
      identifier func;
      @@
      ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
      {
      <+...
      (
        *off = E
      |
        *off += E
      |
        func(..., off, ...)
      |
        E = *off
      )
      ...+>
      }
      
      @ write_no_fpos @
      identifier write_f;
      identifier f, p, s, off;
      type ssize_t, size_t, loff_t;
      @@
      ssize_t write_f(struct file *f, const char *p, size_t s, loff_t *off)
      {
      ... when != off
      }
      
      @ fops0 @
      identifier fops;
      @@
      struct file_operations fops = {
       ...
      };
      
      @ has_llseek depends on fops0 @
      identifier fops0.fops;
      identifier llseek_f;
      @@
      struct file_operations fops = {
      ...
       .llseek = llseek_f,
      ...
      };
      
      @ has_read depends on fops0 @
      identifier fops0.fops;
      identifier read_f;
      @@
      struct file_operations fops = {
      ...
       .read = read_f,
      ...
      };
      
      @ has_write depends on fops0 @
      identifier fops0.fops;
      identifier write_f;
      @@
      struct file_operations fops = {
      ...
       .write = write_f,
      ...
      };
      
      @ has_open depends on fops0 @
      identifier fops0.fops;
      identifier open_f;
      @@
      struct file_operations fops = {
      ...
       .open = open_f,
      ...
      };
      
      // use no_llseek if we call nonseekable_open
      ////////////////////////////////////////////
      @ nonseekable1 depends on !has_llseek && has_open @
      identifier fops0.fops;
      identifier nso ~= "nonseekable_open";
      @@
      struct file_operations fops = {
      ...  .open = nso, ...
      +.llseek = no_llseek, /* nonseekable */
      };
      
      @ nonseekable2 depends on !has_llseek @
      identifier fops0.fops;
      identifier open.open_f;
      @@
      struct file_operations fops = {
      ...  .open = open_f, ...
      +.llseek = no_llseek, /* open uses nonseekable */
      };
      
      // use seq_lseek for sequential files
      /////////////////////////////////////
      @ seq depends on !has_llseek @
      identifier fops0.fops;
      identifier sr ~= "seq_read";
      @@
      struct file_operations fops = {
      ...  .read = sr, ...
      +.llseek = seq_lseek, /* we have seq_read */
      };
      
      // use default_llseek if there is a readdir
      ///////////////////////////////////////////
      @ fops1 depends on !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier readdir_e;
      @@
      // any other fop is used that changes pos
      struct file_operations fops = {
      ... .readdir = readdir_e, ...
      +.llseek = default_llseek, /* readdir is present */
      };
      
      // use default_llseek if at least one of read/write touches f_pos
      /////////////////////////////////////////////////////////////////
      @ fops2 depends on !fops1 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier read.read_f;
      @@
      // read fops use offset
      struct file_operations fops = {
      ... .read = read_f, ...
      +.llseek = default_llseek, /* read accesses f_pos */
      };
      
      @ fops3 depends on !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier write.write_f;
      @@
      // write fops use offset
      struct file_operations fops = {
      ... .write = write_f, ...
      +	.llseek = default_llseek, /* write accesses f_pos */
      };
      
      // Use noop_llseek if neither read nor write accesses f_pos
      ///////////////////////////////////////////////////////////
      
      @ fops4 depends on !fops1 && !fops2 && !fops3 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier read_no_fpos.read_f;
      identifier write_no_fpos.write_f;
      @@
      // write fops use offset
      struct file_operations fops = {
      ...
       .write = write_f,
       .read = read_f,
      ...
      +.llseek = noop_llseek, /* read and write both use no f_pos */
      };
      
      @ depends on has_write && !has_read && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier write_no_fpos.write_f;
      @@
      struct file_operations fops = {
      ... .write = write_f, ...
      +.llseek = noop_llseek, /* write uses no f_pos */
      };
      
      @ depends on has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      identifier read_no_fpos.read_f;
      @@
      struct file_operations fops = {
      ... .read = read_f, ...
      +.llseek = noop_llseek, /* read uses no f_pos */
      };
      
      @ depends on !has_read && !has_write && !fops1 && !fops2 && !has_llseek && !nonseekable1 && !nonseekable2 && !seq @
      identifier fops0.fops;
      @@
      struct file_operations fops = {
      ...
      +.llseek = noop_llseek, /* no read or write fn */
      };
      ===== End semantic patch =====
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Cc: Julia Lawall <julia@diku.dk>
      Cc: Christoph Hellwig <hch@infradead.org>
      6038f373
  28. 09 Sep, 2010 1 commit
  29. 08 Sep, 2010 1 commit
    • Daniel De Graaf's avatar
      xenbus: add missing wakeup in concurrent read/write · 76ce7618
      Daniel De Graaf authored
      
      
      If an application has a dedicated read thread watching xenbus and
      another thread writes an XS_WATCH message that generates a synthetic
      "OK" reply, this reply will be enqueued in the buffer without waking up
      the reader. This can cause a deadlock in the application if it then
      waits for the read thread to receive the queued message.
      Signed-off-by: default avatarDaniel De Graaf <dgdegra@tycho.nsa.gov>
      
      commit e752969f502a511e83f841aa01d6cd332e6d85a0
      Author: Daniel De Graaf <dgdegra@tycho.nsa.gov>
      Date:   Tue Sep 7 11:21:52 2010 -0400
      
          xenbus: fix deadlock in concurrent read/write
      
          If an application has a dedicated read thread watching xenbus and another
          thread writes an XS_WATCH message that generates a synthetic "OK" reply,
          this reply will be enqueued in the buffer without waking up the reader.
      Signed-off-by: default avatarJeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
      76ce7618
  30. 01 Sep, 2010 1 commit
  31. 25 Aug, 2010 1 commit
  32. 26 Jul, 2010 1 commit
  33. 30 Mar, 2010 1 commit
    • Tejun Heo's avatar
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo authored
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      
        http://userweb.kernel.org/~tj/misc/slabh-sweep.py
      
      
      
      The script does the followings.
      
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
        file.
      
      The conversion was done in the following steps.
      
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
         files.
      
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
         necessary.
      
      6. percpu.h was updated not to include slab.h.
      
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
      
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Guess-its-ok-by: default avatarChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
      5a0e3ad6
  34. 04 Oct, 2009 1 commit
  35. 26 Jan, 2009 1 commit