1. 08 Feb, 2020 1 commit
  2. 07 Feb, 2020 1 commit
    • Damien Le Moal's avatar
      fs: New zonefs file system · 8dcc1a9d
      Damien Le Moal authored
      
      
      zonefs is a very simple file system exposing each zone of a zoned block
      device as a file. Unlike a regular file system with zoned block device
      support (e.g. f2fs), zonefs does not hide the sequential write
      constraint of zoned block devices to the user. Files representing
      sequential write zones of the device must be written sequentially
      starting from the end of the file (append only writes).
      
      As such, zonefs is in essence closer to a raw block device access
      interface than to a full featured POSIX file system. The goal of zonefs
      is to simplify the implementation of zoned block device support in
      applications by replacing raw block device file accesses with a richer
      file API, avoiding relying on direct block device file ioctls which may
      be more obscure to developers. One example of this approach is the
      implementation of LSM (log-structured merge) tree structures (such as
      used in RocksDB and LevelDB) on zoned block devices by allowing SSTables
      to be stored in a zone file similarly to a regular file system rather
      than as a range of sectors of a zoned device. The introduction of the
      higher level construct "one file is one zone" can help reducing the
      amount of changes needed in the application as well as introducing
      support for different application programming languages.
      
      Zonefs on-disk metadata is reduced to an immutable super block to
      persistently store a magic number and optional feature flags and
      values. On mount, zonefs uses blkdev_report_zones() to obtain the device
      zone configuration and populates the mount point with a static file tree
      solely based on this information. E.g. file sizes come from the device
      zone type and write pointer offset managed by the device itself.
      
      The zone files created on mount have the following characteristics.
      1) Files representing zones of the same type are grouped together
         under a common sub-directory:
           * For conventional zones, the sub-directory "cnv" is used.
           * For sequential write zones, the sub-directory "seq" is used.
        These two directories are the only directories that exist in zonefs.
        Users cannot create other directories and cannot rename nor delete
        the "cnv" and "seq" sub-directories.
      2) The name of zone files is the number of the file within the zone
         type sub-directory, in order of increasing zone start sector.
      3) The size of conventional zone files is fixed to the device zone size.
         Conventional zone files cannot be truncated.
      4) The size of sequential zone files represent the file's zone write
         pointer position relative to the zone start sector. Truncating these
         files is allowed only down to 0, in which case, the zone is reset to
         rewind the zone write pointer position to the start of the zone, or
         up to the zone size, in which case the file's zone is transitioned
         to the FULL state (finish zone operation).
      5) All read and write operations to files are not allowed beyond the
         file zone size. Any access exceeding the zone size is failed with
         the -EFBIG error.
      6) Creating, deleting, renaming or modifying any attribute of files and
         sub-directories is not allowed.
      7) There are no restrictions on the type of read and write operations
         that can be issued to conventional zone files. Buffered, direct and
         mmap read & write operations are accepted. For sequential zone files,
         there are no restrictions on read operations, but all write
         operations must be direct IO append writes. mmap write of sequential
         files is not allowed.
      
      Several optional features of zonefs can be enabled at format time.
      * Conventional zone aggregation: ranges of contiguous conventional
        zones can be aggregated into a single larger file instead of the
        default one file per zone.
      * File ownership: The owner UID and GID of zone files is by default 0
        (root) but can be changed to any valid UID/GID.
      * File access permissions: the default 640 access permissions can be
        changed.
      
      The mkzonefs tool is used to format zoned block devices for use with
      zonefs. This tool is available on Github at:
      
      git@github.com:damien-lemoal/zonefs-tools.git.
      
      zonefs-tools also includes a test suite which can be run against any
      zoned block device, including null_blk block device created with zoned
      mode.
      
      Example: the following formats a 15TB host-managed SMR HDD with 256 MB
      zones with the conventional zones aggregation feature enabled.
      
      $ sudo mkzonefs -o aggr_cnv /dev/sdX
      $ sudo mount -t zonefs /dev/sdX /mnt
      $ ls -l /mnt/
      total 0
      dr-xr-xr-x 2 root root     1 Nov 25 13:23 cnv
      dr-xr-xr-x 2 root root 55356 Nov 25 13:23 seq
      
      The size of the zone files sub-directories indicate the number of files
      existing for each type of zones. In this example, there is only one
      conventional zone file (all conventional zones are aggregated under a
      single file).
      
      $ ls -l /mnt/cnv
      total 137101312
      -rw-r----- 1 root root 140391743488 Nov 25 13:23 0
      
      This aggregated conventional zone file can be used as a regular file.
      
      $ sudo mkfs.ext4 /mnt/cnv/0
      $ sudo mount -o loop /mnt/cnv/0 /data
      
      The "seq" sub-directory grouping files for sequential write zones has
      in this example 55356 zones.
      
      $ ls -lv /mnt/seq
      total 14511243264
      -rw-r----- 1 root root 0 Nov 25 13:23 0
      -rw-r----- 1 root root 0 Nov 25 13:23 1
      -rw-r----- 1 root root 0 Nov 25 13:23 2
      ...
      -rw-r----- 1 root root 0 Nov 25 13:23 55354
      -rw-r----- 1 root root 0 Nov 25 13:23 55355
      
      For sequential write zone files, the file size changes as data is
      appended at the end of the file, similarly to any regular file system.
      
      $ dd if=/dev/zero of=/mnt/seq/0 bs=4K count=1 conv=notrunc oflag=direct
      1+0 records in
      1+0 records out
      4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000452219 s, 9.1 MB/s
      
      $ ls -l /mnt/seq/0
      -rw-r----- 1 root root 4096 Nov 25 13:23 /mnt/seq/0
      
      The written file can be truncated to the zone size, preventing any
      further write operation.
      
      $ truncate -s 268435456 /mnt/seq/0
      $ ls -l /mnt/seq/0
      -rw-r----- 1 root root 268435456 Nov 25 13:49 /mnt/seq/0
      
      Truncation to 0 size allows freeing the file zone storage space and
      restart append-writes to the file.
      
      $ truncate -s 0 /mnt/seq/0
      $ ls -l /mnt/seq/0
      -rw-r----- 1 root root 0 Nov 25 13:49 /mnt/seq/0
      
      Since files are statically mapped to zones on the disk, the number of
      blocks of a file as reported by stat() and fstat() indicates the size
      of the file zone.
      
      $ stat /mnt/seq/0
        File: /mnt/seq/0
        Size: 0       Blocks: 524288     IO Block: 4096   regular empty file
      Device: 870h/2160d      Inode: 50431       Links: 1
      Access: (0640/-rw-r-----)  Uid: (    0/    root)   Gid: (    0/  root)
      Access: 2019-11-25 13:23:57.048971997 +0900
      Modify: 2019-11-25 13:52:25.553805765 +0900
      Change: 2019-11-25 13:52:25.553805765 +0900
       Birth: -
      
      The number of blocks of the file ("Blocks") in units of 512B blocks
      gives the maximum file size of 524288 * 512 B = 256 MB, corresponding
      to the device zone size in this example. Of note is that the "IO block"
      field always indicates the minimum IO size for writes and corresponds
      to the device physical sector size.
      
      This code contains contributions from:
      * Johannes Thumshirn <jthumshirn@suse.de>,
      * Darrick J. Wong <darrick.wong@oracle.com>,
      * Christoph Hellwig <hch@lst.de>,
      * Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> and
      * Ting Yao <tingyao@hust.edu.cn>.
      Signed-off-by: default avatarDamien Le Moal <damien.lemoal@wdc.com>
      Reviewed-by: default avatarDave Chinner <dchinner@redhat.com>
      8dcc1a9d
  3. 29 Oct, 2019 1 commit
    • Jens Axboe's avatar
      io-wq: small threadpool implementation for io_uring · 771b53d0
      Jens Axboe authored
      
      
      This adds support for io-wq, a smaller and specialized thread pool
      implementation. This is meant to replace workqueues for io_uring. Among
      the reasons for this addition are:
      
      - We can assign memory context smarter and more persistently if we
        manage the life time of threads.
      
      - We can drop various work-arounds we have in io_uring, like the
        async_list.
      
      - We can implement hashed work insertion, to manage concurrency of
        buffered writes without needing a) an extra workqueue, or b)
        needlessly making the concurrency of said workqueue very low
        which hurts performance of multiple buffered file writers.
      
      - We can implement cancel through signals, for cancelling
        interruptible work like read/write (or send/recv) to/from sockets.
      
      - We need the above cancel for being able to assign and use file tables
        from a process.
      
      - We can implement a more thorough cancel operation in general.
      
      - We need it to move towards a syslet/threadlet model for even faster
        async execution. For that we need to take ownership of the used
        threads.
      
      This list is just off the top of my head. Performance should be the
      same, or better, at least that's what I've seen in my testing. io-wq
      supports basic NUMA functionality, setting up a pool per node.
      
      io-wq hooks up to the scheduler schedule in/out just like workqueue
      and uses that to drive the need for more/less workers.
      Acked-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      771b53d0
  4. 24 Aug, 2019 1 commit
    • Gao Xiang's avatar
      erofs: move erofs out of staging · 47e4937a
      Gao Xiang authored
      
      
      EROFS filesystem has been merged into linux-staging for a year.
      
      EROFS is designed to be a better solution of saving extra storage
      space with guaranteed end-to-end performance for read-only files
      with the help of reduced metadata, fixed-sized output compression
      and decompression inplace technologies.
      
      In the past year, EROFS was greatly improved by many people as
      a staging driver, self-tested, betaed by a large number of our
      internal users, successfully applied to almost all in-service
      HUAWEI smartphones as the part of EMUI 9.1 and proven to be stable
      enough to be moved out of staging.
      
      EROFS is a self-contained filesystem driver. Although there are
      still some TODOs to be more generic, we have a dedicated team
      actively keeping on working on EROFS in order to make it better
      with the evolution of Linux kernel as the other in-kernel filesystems.
      
      As Pavel suggested, it's better to do as one commit since git
      can do moves and all histories will be saved in this way.
      
      Let's promote it from staging and enhance it more actively as
      a "real" part of kernel for more wider scenarios!
      
      Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
      Cc: Alexander Viro <viro@zeniv.linux.org.uk>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Stephen Rothwell <sfr@canb.auug.org.au>
      Cc: Theodore Ts'o <tytso@mit.edu>
      Cc: Pavel Machek <pavel@denx.de>
      Cc: David Sterba <dsterba@suse.cz>
      Cc: Amir Goldstein <amir73il@gmail.com>
      Cc: Christoph Hellwig <hch@infradead.org>
      Cc: Darrick J . Wong <darrick.wong@oracle.com>
      Cc: Dave Chinner <david@fromorbit.com>
      Cc: Jaegeuk Kim <jaegeuk@kernel.org>
      Cc: Jan Kara <jack@suse.cz>
      Cc: Richard Weinberger <richard@nod.at>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Chao Yu <yuchao0@huawei.com>
      Cc: Miao Xie <miaoxie@huawei.com>
      Cc: Li Guifu <bluce.liguifu@huawei.com>
      Cc: Fang Wei <fangwei1@huawei.com>
      Signed-off-by: default avatarGao Xiang <gaoxiang25@huawei.com>
      Link: https://lore.kernel.org/r/20190822213659.5501-1-hsiangkao@aol.com
      
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      47e4937a
  5. 28 Jul, 2019 1 commit
  6. 05 Jul, 2019 1 commit
  7. 21 May, 2019 1 commit
  8. 25 Apr, 2019 1 commit
    • Gabriel Krisman Bertazi's avatar
      unicode: introduce UTF-8 character database · 955405d1
      Gabriel Krisman Bertazi authored
      The decomposition and casefolding of UTF-8 characters are described in a
      prefix tree in utf8data.h, which is a generate from the Unicode
      Character Database (UCD), published by the Unicode Consortium, and
      should not be edited by hand.  The structures in utf8data.h are meant to
      be used for lookup operations by the unicode subsystem, when decoding a
      utf-8 string.
      
      mkutf8data.c is the source for a program that generates utf8data.h. It
      was written by Olaf Weber from SGI and originally proposed to be merged
      into Linux in 2014.  The original proposal performed the compatibility
      decomposition, NFKD, but the current version was modified by me to do
      canonical decomposition, NFD, as suggested by the community.  The
      changes from the original submission are:
      
        * Rebase to mainline.
        * Fix out-of-tree-build.
        * Update makefile to build 11.0.0 ucd files.
        * drop references to xfs.
        * Convert NFKD to NFD.
        * Merge back robustness fixes from original patch. Requested by
          Dave Chinner.
      
      The original submission is archived at:
      
      <https://linux-xfs.oss.sgi.narkive.com/Xx10wjVY/rfc-unicode-utf-8-support-for-xfs
      
      >
      
      The utf8data.h file can be regenerated using the instructions in
      fs/unicode/README.utf8data.
      
      - Notes on the update from 8.0.0 to 11.0:
      
      The structure of the ucd files and special cases have not experienced
      any changes between versions 8.0.0 and 11.0.0.  8.0.0 saw the addition
      of Cherokee LC characters, which is an interesting case for
      case-folding.  The update is accompanied by new tests on the test_ucd
      module to catch specific cases.  No changes to mkutf8data script were
      required for the updates.
      Signed-off-by: default avatarGabriel Krisman Bertazi <krisman@collabora.co.uk>
      Signed-off-by: default avatarTheodore Ts'o <tytso@mit.edu>
      955405d1
  9. 28 Feb, 2019 1 commit
    • David Howells's avatar
      vfs: Add configuration parser helpers · 31d921c7
      David Howells authored
      
      
      Because the new API passes in key,value parameters, match_token() cannot be
      used with it.  Instead, provide three new helpers to aid with parsing:
      
       (1) fs_parse().  This takes a parameter and a simple static description of
           all the parameters and maps the key name to an ID.  It returns 1 on a
           match, 0 on no match if unknowns should be ignored and some other
           negative error code on a parse error.
      
           The parameter description includes a list of key names to IDs, desired
           parameter types and a list of enumeration name -> ID mappings.
      
           [!] Note that for the moment I've required that the key->ID mapping
           array is expected to be sorted and unterminated.  The size of the
           array is noted in the fsconfig_parser struct.  This allows me to use
           bsearch(), but I'm not sure any performance gain is worth the hassle
           of requiring people to keep the array sorted.
      
           The parameter type array is sized according to the number of parameter
           IDs and is indexed directly.  The optional enum mapping array is an
           unterminated, unsorted list and the size goes into the fsconfig_parser
           struct.
      
           The function can do some additional things:
      
      	(a) If it's not ambiguous and no value is given, the prefix "no" on
      	    a key name is permitted to indicate that the parameter should
      	    be considered negatory.
      
      	(b) If the desired type is a single simple integer, it will perform
      	    an appropriate conversion and store the result in a union in
      	    the parse result.
      
      	(c) If the desired type is an enumeration, {key ID, name} will be
      	    looked up in the enumeration list and the matching value will
      	    be stored in the parse result union.
      
      	(d) Optionally generate an error if the key is unrecognised.
      
           This is called something like:
      
      	enum rdt_param {
      		Opt_cdp,
      		Opt_cdpl2,
      		Opt_mba_mpbs,
      		nr__rdt_params
      	};
      
      	const struct fs_parameter_spec rdt_param_specs[nr__rdt_params] = {
      		[Opt_cdp]	= { fs_param_is_bool },
      		[Opt_cdpl2]	= { fs_param_is_bool },
      		[Opt_mba_mpbs]	= { fs_param_is_bool },
      	};
      
      	const const char *const rdt_param_keys[nr__rdt_params] = {
      		[Opt_cdp]	= "cdp",
      		[Opt_cdpl2]	= "cdpl2",
      		[Opt_mba_mpbs]	= "mba_mbps",
      	};
      
      	const struct fs_parameter_description rdt_parser = {
      		.name		= "rdt",
      		.nr_params	= nr__rdt_params,
      		.keys		= rdt_param_keys,
      		.specs		= rdt_param_specs,
      		.no_source	= true,
      	};
      
      	int rdt_parse_param(struct fs_context *fc,
      			    struct fs_parameter *param)
      	{
      		struct fs_parse_result parse;
      		struct rdt_fs_context *ctx = rdt_fc2context(fc);
      		int ret;
      
      		ret = fs_parse(fc, &rdt_parser, param, &parse);
      		if (ret < 0)
      			return ret;
      
      		switch (parse.key) {
      		case Opt_cdp:
      			ctx->enable_cdpl3 = true;
      			return 0;
      		case Opt_cdpl2:
      			ctx->enable_cdpl2 = true;
      			return 0;
      		case Opt_mba_mpbs:
      			ctx->enable_mba_mbps = true;
      			return 0;
      		}
      
      		return -EINVAL;
      	}
      
       (2) fs_lookup_param().  This takes a { dirfd, path, LOOKUP_EMPTY? } or
           string value and performs an appropriate path lookup to convert it
           into a path object, which it will then return.
      
           If the desired type was a blockdev, the type of the looked up inode
           will be checked to make sure it is one.
      
           This can be used like:
      
      	enum foo_param {
      		Opt_source,
      		nr__foo_params
      	};
      
      	const struct fs_parameter_spec foo_param_specs[nr__foo_params] = {
      		[Opt_source]	= { fs_param_is_blockdev },
      	};
      
      	const char *char foo_param_keys[nr__foo_params] = {
      		[Opt_source]	= "source",
      	};
      
      	const struct constant_table foo_param_alt_keys[] = {
      		{ "device",	Opt_source },
      	};
      
      	const struct fs_parameter_description foo_parser = {
      		.name		= "foo",
      		.nr_params	= nr__foo_params,
      		.nr_alt_keys	= ARRAY_SIZE(foo_param_alt_keys),
      		.keys		= foo_param_keys,
      		.alt_keys	= foo_param_alt_keys,
      		.specs		= foo_param_specs,
      	};
      
      	int foo_parse_param(struct fs_context *fc,
      			    struct fs_parameter *param)
      	{
      		struct fs_parse_result parse;
      		struct foo_fs_context *ctx = foo_fc2context(fc);
      		int ret;
      
      		ret = fs_parse(fc, &foo_parser, param, &parse);
      		if (ret < 0)
      			return ret;
      
      		switch (parse.key) {
      		case Opt_source:
      			return fs_lookup_param(fc, &foo_parser, param,
      					       &parse, &ctx->source);
      		default:
      			return -EINVAL;
      		}
      	}
      
       (3) lookup_constant().  This takes a table of named constants and looks up
           the given name within it.  The table is expected to be sorted such
           that bsearch() be used upon it.
      
           Possibly I should require the table be terminated and just use a
           for-loop to scan it instead of using bsearch() to reduce hassle.
      
           Tables look something like:
      
      	static const struct constant_table bool_names[] = {
      		{ "0",		false },
      		{ "1",		true },
      		{ "false",	false },
      		{ "no",		false },
      		{ "true",	true },
      		{ "yes",	true },
      	};
      
           and a lookup is done with something like:
      
      	b = lookup_constant(bool_names, param->string, -1);
      
      Additionally, optional validation routines for the parameter description
      are provided that can be enabled at compile time.  A later patch will
      invoke these when a filesystem is registered.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      31d921c7
  10. 06 Feb, 2019 1 commit
  11. 11 Jun, 2018 1 commit
    • Linus Torvalds's avatar
      autofs: remove left-over autofs4 stubs · a2225d93
      Linus Torvalds authored
      
      
      There's no need to retain the fs/autofs4 directory for backward
      compatibility.
      
      Adding an AUTOFS4_FS fragment to the autofs Kconfig and a module alias
      for autofs4 is sufficient for almost all cases. Not keeping fs/autofs4
      remnants will prevent "insmod <path>/autofs4/autofs4.ko" from working
      but this shouldn't be used in automation scripts rather than
      modprobe(8).
      
      There were some comments about things to look out for with the module
      rename in the fs/autofs4/Kconfig that is removed by this patch, see the
      commit patch if you are interested.
      
      One potential problem with this change is that when the
      fs/autofs/Kconfig fragment for AUTOFS4_FS is removed any AUTOFS4_FS
      entries will be removed from the kernel config, resulting in no autofs
      file system being built if there is no AUTOFS_FS entry also.
      
      This would have also happened if the fs/autofs4 remnants had remained
      and is most likely to be a problem with automated builds.
      
      Please check your build configurations before the removal which will
      occur after the next couple of kernel releases.
      Acked-by: default avatarIan Kent <raven@themaw.net>
      [ With edits and commit message from Ian Kent ]
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      a2225d93
  12. 08 Jun, 2018 2 commits
  13. 22 May, 2018 1 commit
    • Dan Williams's avatar
      mm: introduce MEMORY_DEVICE_FS_DAX and CONFIG_DEV_PAGEMAP_OPS · e7638488
      Dan Williams authored
      
      
      In preparation for fixing dax-dma-vs-unmap issues, filesystems need to
      be able to rely on the fact that they will get wakeups on dev_pagemap
      page-idle events. Introduce MEMORY_DEVICE_FS_DAX and
      generic_dax_page_free() as common indicator / infrastructure for dax
      filesytems to require. With this change there are no users of the
      MEMORY_DEVICE_HOST designation, so remove it.
      
      The HMM sub-system extended dev_pagemap to arrange a callback when a
      dev_pagemap managed page is freed. Since a dev_pagemap page is free /
      idle when its reference count is 1 it requires an additional branch to
      check the page-type at put_page() time. Given put_page() is a hot-path
      we do not want to incur that check if HMM is not in use, so a static
      branch is used to avoid that overhead when not necessary.
      
      Now, the FS_DAX implementation wants to reuse this mechanism for
      receiving dev_pagemap ->page_free() callbacks. Rework the HMM-specific
      static-key into a generic mechanism that either HMM or FS_DAX code paths
      can enable.
      
      For ARCH=um builds, and any other arch that lacks ZONE_DEVICE support,
      care must be taken to compile out the DEV_PAGEMAP_OPS infrastructure.
      However, we still need to support FS_DAX in the FS_DAX_LIMITED case
      implemented by the s390/dcssblk driver.
      
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Cc: Michal Hocko <mhocko@suse.com>
      Reported-by: default avatarkbuild test robot <lkp@intel.com>
      Reported-by: default avatarThomas Meyer <thomas@m3y3r.de>
      Reported-by: default avatarDave Jiang <dave.jiang@intel.com>
      Cc: "Jérôme Glisse" <jglisse@redhat.com>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarDan Williams <dan.j.williams@intel.com>
      e7638488
  14. 27 Apr, 2018 1 commit
  15. 16 Apr, 2018 1 commit
  16. 20 Jan, 2018 1 commit
    • Dan Williams's avatar
      dax: require 'struct page' by default for filesystem dax · 569d0365
      Dan Williams authored
      
      
      If a dax buffer from a device that does not map pages is passed to
      read(2) or write(2) as a target for direct-I/O it triggers SIGBUS. If
      gdb attempts to examine the contents of a dax buffer from a device that
      does not map pages it triggers SIGBUS. If fork(2) is called on a process
      with a dax mapping from a device that does not map pages it triggers
      SIGBUS. 'struct page' is required otherwise several kernel code paths
      break in surprising ways. Disable filesystem-dax on devices that do not
      map pages.
      
      In addition to needing pfn_to_page() to be valid we also require devmap
      pages.  We need this to detect dax pages in the get_user_pages_fast()
      path and so that we can stop managing the VM_MIXEDMAP flag. For DAX
      drivers that have not supported get_user_pages() to date we allow them
      to opt-in to supporting DAX with the CONFIG_FS_DAX_LIMITED configuration
      option which requires ->direct_access() to return pfn_t_special() pfns.
      This leaves DAX support in brd disabled and scheduled for removal.
      
      Note that when the initial dax support was being merged a few years back
      there was concern that struct page was unsuitable for use with next
      generation persistent memory devices. The theoretical concern was that
      struct page access, being such a hotly used data structure in the
      kernel, would lead to media wear out. While that was a reasonable
      conservative starting position it has not held true in practice. We have
      long since committed to using devm_memremap_pages() to support higher
      order kernel functionality that needs get_user_pages() and
      pfn_to_page().
      
      
      Cc: Jeff Moyer <jmoyer@redhat.com>
      Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Michael Ellerman <mpe@ellerman.id.au>
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
      Reviewed-by: default avatarJan Kara <jack@suse.cz>
      Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarGerald Schaefer <gerald.schaefer@de.ibm.com>
      Signed-off-by: default avatarDan Williams <dan.j.williams@intel.com>
      569d0365
  17. 01 Jan, 2018 1 commit
  18. 28 Nov, 2017 1 commit
  19. 12 Jul, 2017 1 commit
  20. 08 May, 2017 1 commit
    • Dan Williams's avatar
      block, dax: move "select DAX" from BLOCK to FS_DAX · ef510424
      Dan Williams authored
      
      
      For configurations that do not enable DAX filesystems or drivers, do not
      require the DAX core to be built.
      
      Given that the 'direct_access' method has been removed from
      'block_device_operations', we can also go ahead and remove the
      block-related dax helper functions from fs/block_dev.c to
      drivers/dax/super.c. This keeps dax details out of the block layer and
      lets the DAX core be built as a module in the FS_DAX=n case.
      
      Filesystems need to include dax.h to call bdev_dax_supported().
      
      Cc: linux-xfs@vger.kernel.org
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: "Theodore Ts'o" <tytso@mit.edu>
      Cc: Matthew Wilcox <mawilcox@microsoft.com>
      Cc: Alexander Viro <viro@zeniv.linux.org.uk>
      Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
      Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
      Reviewed-by: default avatarJan Kara <jack@suse.com>
      Reported-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
      Signed-off-by: default avatarDan Williams <dan.j.williams@intel.com>
      ef510424
  21. 25 Jan, 2017 1 commit
  22. 15 Dec, 2016 1 commit
    • Christoph Hellwig's avatar
      logfs: remove from tree · 1d0fd57a
      Christoph Hellwig authored
      
      
      Logfs was introduced to the kernel in 2009, and hasn't seen any non
      drive-by changes since 2012, while having lots of unsolved issues
      including the complete lack of error handling, with more and more
      issues popping up without any fixes.
      
      The logfs.org domain has been bouncing from a mail, and the maintainer
      on the non-logfs.org domain hasn't repsonded to past queries either.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      1d0fd57a
  23. 08 Nov, 2016 1 commit
  24. 08 Oct, 2016 1 commit
  25. 22 Sep, 2016 1 commit
  26. 15 Jul, 2016 1 commit
  27. 20 Jun, 2016 1 commit
    • Christoph Hellwig's avatar
      fs: introduce iomap infrastructure · ae259a9c
      Christoph Hellwig authored
      
      
      Add infrastructure for multipage buffered writes.  This is implemented
      using an main iterator that applies an actor function to a range that
      can be written.
      
      This infrastucture is used to implement a buffered write helper, one
      to zero file ranges and one to implement the ->page_mkwrite VM
      operations.  All of them borrow a fair amount of code from fs/buffers.
      for now by using an internal version of __block_write_begin that
      gets passed an iomap and builds the corresponding buffer head.
      
      The file system is gets a set of paired ->iomap_begin and ->iomap_end
      calls which allow it to map/reserve a range and get a notification
      once the write code is finished with it.
      
      Based on earlier code from Dave Chinner.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Reviewed-by: default avatarBob Peterson <rpeterso@redhat.com>
      Signed-off-by: default avatarDave Chinner <david@fromorbit.com>
      
      ae259a9c
  28. 19 May, 2016 1 commit
    • Jan Kara's avatar
      dax: Make huge page handling depend of CONFIG_BROKEN · 348e967a
      Jan Kara authored
      
      
      Currently the handling of huge pages for DAX is racy. For example the
      following can happen:
      
      CPU0 (THP write fault)			CPU1 (normal read fault)
      
      __dax_pmd_fault()			__dax_fault()
        get_block(inode, block, &bh, 0) -> not mapped
      					get_block(inode, block, &bh, 0)
      					  -> not mapped
        if (!buffer_mapped(&bh) && write)
          get_block(inode, block, &bh, 1) -> allocates blocks
        truncate_pagecache_range(inode, lstart, lend);
      					dax_load_hole();
      
      This results in data corruption since process on CPU1 won't see changes
      into the file done by CPU0.
      
      The race can happen even if two normal faults race however with THP the
      situation is even worse because the two faults don't operate on the same
      entries in the radix tree and we want to use these entries for
      serialization. So make THP support in DAX code depend on CONFIG_BROKEN
      for now.
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      Signed-off-by: default avatarRoss Zwisler <ross.zwisler@linux.intel.com>
      348e967a
  29. 18 Mar, 2016 1 commit
    • Jaegeuk Kim's avatar
      fs crypto: move per-file encryption from f2fs tree to fs/crypto · 0b81d077
      Jaegeuk Kim authored
      
      
      This patch adds the renamed functions moved from the f2fs crypto files.
      
      1. definitions for per-file encryption used by ext4 and f2fs.
      
      2. crypto.c for encrypt/decrypt functions
       a. IO preparation:
        - fscrypt_get_ctx / fscrypt_release_ctx
       b. before IOs:
        - fscrypt_encrypt_page
        - fscrypt_decrypt_page
        - fscrypt_zeroout_range
       c. after IOs:
        - fscrypt_decrypt_bio_pages
        - fscrypt_pullback_bio_page
        - fscrypt_restore_control_page
      
      3. policy.c supporting context management.
       a. For ioctls:
        - fscrypt_process_policy
        - fscrypt_get_policy
       b. For context permission
        - fscrypt_has_permitted_context
        - fscrypt_inherit_context
      
      4. keyinfo.c to handle permissions
        - fscrypt_get_encryption_info
        - fscrypt_free_encryption_info
      
      5. fname.c to support filename encryption
       a. general wrapper functions
        - fscrypt_fname_disk_to_usr
        - fscrypt_fname_usr_to_disk
        - fscrypt_setup_filename
        - fscrypt_free_filename
      
       b. specific filename handling functions
        - fscrypt_fname_alloc_buffer
        - fscrypt_fname_free_buffer
      
      6. Makefile and Kconfig
      
      Cc: Al Viro <viro@ftp.linux.org.uk>
      Signed-off-by: default avatarMichael Halcrow <mhalcrow@google.com>
      Signed-off-by: default avatarIldar Muslukhov <ildarm@google.com>
      Signed-off-by: default avatarUday Savagaonkar <savagaon@google.com>
      Signed-off-by: default avatarTheodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarArnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarJaegeuk Kim <jaegeuk@kernel.org>
      0b81d077
  30. 16 Jan, 2016 1 commit
  31. 17 Nov, 2015 1 commit
    • Dan Williams's avatar
      dax: disable pmd mappings · ee82c9ed
      Dan Williams authored
      
      
      While dax pmd mappings are functional in the nominal path they trigger
      kernel crashes in the following paths:
      
       BUG: unable to handle kernel paging request at ffffea0004098000
       IP: [<ffffffff812362f7>] follow_trans_huge_pmd+0x117/0x3b0
       [..]
       Call Trace:
        [<ffffffff811f6573>] follow_page_mask+0x2d3/0x380
        [<ffffffff811f6708>] __get_user_pages+0xe8/0x6f0
        [<ffffffff811f7045>] get_user_pages_unlocked+0x165/0x1e0
        [<ffffffff8106f5b1>] get_user_pages_fast+0xa1/0x1b0
      
       kernel BUG at arch/x86/mm/gup.c:131!
       [..]
       Call Trace:
        [<ffffffff8106f34c>] gup_pud_range+0x1bc/0x220
        [<ffffffff8106f634>] get_user_pages_fast+0x124/0x1b0
      
       BUG: unable to handle kernel paging request at ffffea0004088000
       IP: [<ffffffff81235f49>] copy_huge_pmd+0x159/0x350
       [..]
       Call Trace:
        [<ffffffff811fad3c>] copy_page_range+0x34c/0x9f0
        [<ffffffff810a0daf>] copy_process+0x1b7f/0x1e10
        [<ffffffff810a11c1>] _do_fork+0x91/0x590
      
      All of these paths are interpreting a dax pmd mapping as a transparent
      huge page and making the assumption that the pfn is covered by the
      memmap, i.e. that the pfn has an associated struct page.  PTE mappings
      do not suffer the same fate since they have the _PAGE_SPECIAL flag to
      cause the gup path to fault.  We can do something similar for the PMD
      path, or otherwise defer pmd support for cases where a struct page is
      available.  For now, 4.4-rc and -stable need to disable dax pmd support
      by default.
      
      For development the "depends on BROKEN" line can be removed from
      CONFIG_FS_DAX_PMD.
      
      Cc: <stable@vger.kernel.org>
      Cc: Jan Kara <jack@suse.com>
      Cc: Dave Chinner <david@fromorbit.com>
      Cc: Matthew Wilcox <willy@linux.intel.com>
      Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Reported-by: default avatarRoss Zwisler <ross.zwisler@linux.intel.com>
      Signed-off-by: default avatarDan Williams <dan.j.williams@intel.com>
      ee82c9ed
  32. 16 Nov, 2015 1 commit
    • Jeff Layton's avatar
      locks: Allow disabling mandatory locking at compile time · 9e8925b6
      Jeff Layton authored
      
      
      Mandatory locking appears to be almost unused and buggy and there
      appears no real interest in doing anything with it.  Since effectively
      no one uses the code and since the code is buggy let's allow it to be
      disabled at compile time.  I would just suggest removing the code but
      undoubtedly that will break some piece of userspace code somewhere.
      
      For the distributions that don't care about this piece of code
      this gives a nice starting point to make mandatory locking go away.
      
      Cc: Benjamin Coddington <bcodding@redhat.com>
      Cc: Dmitry Vyukov <dvyukov@google.com>
      Cc: Jeff Layton <jeff.layton@primarydata.com>
      Cc: J. Bruce Fields <bfields@fieldses.org>
      Signed-off-by: default avatar"Eric W. Biederman" <ebiederm@xmission.com>
      Signed-off-by: default avatarJeff Layton <jeff.layton@primarydata.com>
      9e8925b6
  33. 03 Oct, 2015 1 commit
  34. 23 Jul, 2015 1 commit
    • Jan Kara's avatar
      fs: Remove ext3 filesystem driver · c290ea01
      Jan Kara authored
      
      
      The functionality of ext3 is fully supported by ext4 driver. Major
      distributions (SUSE, RedHat) already use ext4 driver to handle ext3
      filesystems for quite some time. There is some ugliness in mm resulting
      from jbd cleaning buffers in a dirty page without cleaning page dirty
      bit and also support for buffer bouncing in the block layer when stable
      pages are required is there only because of jbd. So let's remove the
      ext3 driver. This saves us some 28k lines of duplicated code.
      Acked-by: default avatarTheodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      c290ea01
  35. 10 Apr, 2015 1 commit
  36. 17 Feb, 2015 2 commits
  37. 05 Jan, 2015 1 commit
  38. 23 Oct, 2014 1 commit
    • Miklos Szeredi's avatar
      overlay filesystem · e9be9d5e
      Miklos Szeredi authored
      
      
      Overlayfs allows one, usually read-write, directory tree to be
      overlaid onto another, read-only directory tree.  All modifications
      go to the upper, writable layer.
      
      This type of mechanism is most often used for live CDs but there's a
      wide variety of other uses.
      
      The implementation differs from other "union filesystem"
      implementations in that after a file is opened all operations go
      directly to the underlying, lower or upper, filesystems.  This
      simplifies the implementation and allows native performance in these
      cases.
      
      The dentry tree is duplicated from the underlying filesystems, this
      enables fast cached lookups without adding special support into the
      VFS.  This uses slightly more memory than union mounts, but dentries
      are relatively small.
      
      Currently inodes are duplicated as well, but it is a possible
      optimization to share inodes for non-directories.
      
      Opening non directories results in the open forwarded to the
      underlying filesystem.  This makes the behavior very similar to union
      mounts (with the same limitations vs. fchmod/fchown on O_RDONLY file
      descriptors).
      
      Usage:
      
        mount -t overlayfs overlayfs -olowerdir=/lower,upperdir=/upper/upper,workdir=/upper/work /overlay
      
      The following cotributions have been folded into this patch:
      
      Neil Brown <neilb@suse.de>:
       - minimal remount support
       - use correct seek function for directories
       - initialise is_real before use
       - rename ovl_fill_cache to ovl_dir_read
      
      Felix Fietkau <nbd@openwrt.org>:
       - fix a deadlock in ovl_dir_read_merged
       - fix a deadlock in ovl_remove_whiteouts
      
      Erez Zadok <ezk@fsl.cs.sunysb.edu>
       - fix cleanup after WARN_ON
      
      Sedat Dilek <sedat.dilek@googlemail.com>
       - fix up permission to confirm to new API
      
      Robin Dong <hao.bigrat@gmail.com>
       - fix possible leak in ovl_new_inode
       - create new inode in ovl_link
      
      Andy Whitcroft <apw@canonical.com>
       - switch to __inode_permission()
       - copy up i_uid/i_gid from the underlying inode
      
      AV:
       - ovl_copy_up_locked() - dput(ERR_PTR(...)) on two failure exits
       - ovl_clear_empty() - one failure exit forgetting to do unlock_rename(),
         lack of check for udir being the parent of upper, dropping and regaining
         the lock on udir (which would require _another_ check for parent being
         right).
       - bogus d_drop() in copyup and rename [fix from your mail]
       - copyup/remove and copyup/rename races [fix from your mail]
       - ovl_dir_fsync() leaving ERR_PTR() in ->realfile
       - ovl_entry_free() is pointless - it's just a kfree_rcu()
       - fold ovl_do_lookup() into ovl_lookup()
       - manually assigning ->d_op is wrong.  Just use ->s_d_op.
       [patches picked from Miklos]:
       * copyup/remove and copyup/rename races
       * bogus d_drop() in copyup and rename
      
      Also thanks to the following people for testing and reporting bugs:
      
        Jordi Pujol <jordipujolp@gmail.com>
        Andy Whitcroft <apw@canonical.com>
        Michal Suchanek <hramrach@centrum.cz>
        Felix Fietkau <nbd@openwrt.org>
        Erez Zadok <ezk@fsl.cs.sunysb.edu>
        Randy Dunlap <rdunlap@xenotime.net>
      Signed-off-by: default avatarMiklos Szeredi <mszeredi@suse.cz>
      e9be9d5e