2020-03-05 Pádraig Brady version 8.32 * NEWS: Record release date. 2020-03-04 Pádraig Brady tests: don't rely on system env(1) being present * tests/misc/env-S.pl: `env -i env` will call the system env due to the path being cleared, so pass the absolute path of our env binary under test to avoid that. This was seen to be an issue on Guix where /usr/bin/env was not available. basenc: avoid undefined behaviour in z85 processing * src/basenc.c (z85_decode_ctx_init): Ensure we're working with unsigned, as otherwise ubsan triggers with: src/basenc.c:767:18: runtime error: signed integer overflow: 43 * 52200625 cannot be represented in type 'int' (z85_encode): Likewise to avoid the usban error: src/basenc.c:630:26: runtime error: left shift of 134 by 24 places cannot be represented in type 'int' 2020-03-01 Pádraig Brady tests: avoid a false failure on OpenIndiana 11 * tests/misc/timeout-parameters.sh: Split the large timeout handling to ... * tests/misc/timeout-large-parameters.sh: ... here, so that the 3 second delay is contained in its own test, and if the test is skipped due invalid handling within timeout(1), it will be more apparent. Also adjust the check so we skip whenever the kernel timer fires immediately, to handle the buggy OpenIndiana 11 kernel also. Reported by Bruno Haible. tests: avoid a hang on GNU/Hurd from 2019 * tests/du/8gb.sh: Add a timeout around: `dd bs=1 seek=8G of=big < /dev/null` tests: use bash in some scripts to avoid false failures * init.cfg (require_bash_as_SHELL_): A new function to replace SHELL for the current test, with bash if available. This is useful on OpenIndiana 11 where /bin/sh was seen to have races in handling of SIGPIPE. * tests/misc/seq-epipe.sh: Use the new function to enforce bash. * tests/misc/env-signal-handler.sh: Likewise. Reported by Bruno Haible tests: improve test coverage for ls stat checks * tests/ls/stat-free-color.sh: Check for the availability of various stat calls individually, and add statx() and fstatat64() to the list to check. Fix the stat counting logic to ignore lines like "+++ exited with 0 +++". * tests/ls/stat-free-symlinks.sh: Check syscalls other than stat(). 2020-03-01 Bruno Haible tests: enable 4 more tests to be executed on FreeBSD * init.cfg (gcc_shared_libs_): New variable. (gcc_shared_): Use it, instead of hardcoding -ldl. (require_gcc_shared_): Determine the suitable value for gcc_shared_libs_. 2020-02-29 Pádraig Brady tests: fix incorrect `|| fail` pattern in tests * tests/ls/stat-free-symlinks.sh: s/|| fail/|| fail=1/. * tests/misc/tee.sh: Likewise. * tests/touch/relative.sh: Likewise. * cfg.mk (sc_prohibit_or_fail): A new syntax-check to avoid this. 2020-02-29 Pádraig Brady tests: avoid false failures on darwin 19.2.0 With these adjustments, all tests pass on macOS Catalina. * tests/dd/sparse.sh: Adjust so that systems like apfs that don't create holes < 16 MiB do not fail erroneously. * tests/touch/trailing-slash.sh: Darwin was seen to dereference symlinks to files when given a trailing slash, so avoid that particular case. 2020-02-29 Bruno Haible tests: fix test failure on FreeBSD 12 * tests/misc/csplit-io-err.sh: Limit the effect of the fwrite override to streams != stderr, as fwrite is in the error() path there. 2020-02-27 Jan Nieuwenhuizen build: once again distribute .tar.gz files * configure.ac: Reenable distribution of gzip-compressed tarballs, for Guix bootstrapping reasons as discussed at: https://lists.gnu.org/r/coreutils/2020-02/msg00042.html * THANKS.in: Remove me, as now a committer. * NEWS (Build-related): Mention this. 2020-02-27 Pádraig Brady maint: ensure .deps/ in the project root is ignored by git * .gitignore: s|*/.deps/|.deps| doc: remove older ChangeLog items * Makefile.am: Update the oldest documented version to 8.23 which is now about 5 years old. 2020-02-27 Colin Watson ls: issue error message on removed directory If the current directory has been removed, then "ls" confusingly produced no output and no error message, indistinguishable from running on an empty directory. * src/ls.c (print_dir): Report ENOENT on GNU/Linux if readdir finds no directory entries at all, not even "." or "..", and a recheck with the getdents syscall returns ENOENT. We recheck with getdents() as POSIX states that "The directory entries for dot and dot-dot are optional". * tests/ls/removed-directory.sh: New file. * tests/local.mk (all_tests): Add new test. * NEWS: Mention the change in behavior. Reported by Owen Thomas. 2020-02-25 Pádraig Brady build: update to latest gnulib * bootstrap.conf: Adjust for changes to fchmodat and fchownat, which are now separated from chmodat and chownat respectively. b2sum: sync better with upstream * src/blake2/blake2-impl.h: Sync load16() implementation, which doesn't change code generation. Also leverage (builtin) memcpy to more efficiently move data on little endian systems, giving a 2% win with GCC 9.2.1 on an i3-2310M. factor: sync longlong.h adjustments from upstream * src/longlong.h: Sync changes from: https://gmplib.org/repo/gmp/log/tip/longlong.h mips64: Provide r6 asm code as default expression yields. arm32: Define sub_ddmmss separately for non-thumb (no rsc instruction). powerpc: Add "CLOBBER" descriptions for some registers. x86: Fix criterion for when to use mulx in umul_ppmm. stat,tail: sync file system constants from the linux kernel * src/stat.c: Add magic constants for "binderfs", "dma-buf-fs", "erofs", "ppc-cmm-fs", and "z3fold". * NEWS: Mention the improvement. 2020-02-24 Pádraig Brady uniq: avoid strcoll() to improve performance and consistency strcoll() is only significant to uniq(1) if it returns 0, and it generally only does so with buggy locales or mismatched locales and data. Some systems may have strcoll() return 0 for equivalent normalized unicode forms, but for consistency across platforms strcoll() is avoided. The various cases are defined in the new test. This is consistent with newer POSIX standards as discussed at: https://www.austingroupbugs.net/view.php?id=963 * src/uniq.c: s/xstrcoll/memcmp/. * tests/local.mk: Reference the new test. * tests/misc/uniq-collate.sh: Add a new test. * NEWS: Mention the change in behavior. Fixes https://bugs.gnu.org/38627 2020-02-15 Pádraig Brady doc: clarify that '%a' stat format outputs mode bits * src/stat.c (usage): Mention permission bits rather than "access" so there is no confusion with ACLs etc. Also indicate we output the file type with '%A'. * doc/coreutils.texi (stat invocation): Likewise. Also indicate '%A' is similar to `ls -ld` output. Addresses https://bugs.gnu.org/39613 2020-02-10 Pádraig Brady tests: fix test for symlink * tests/cp/preserve-gid.sh: s/-l/-L/. Reported by Kamil Dudka 2020-02-09 Kamil Dudka tests: ensure tests/cp/preserve-gid.sh works with single binary * tests/cp/preserve-gid.sh: If configured with --enable-single-binary copy the coreutils single binary, instead of the cp one-line launcher. Discussed at https://bugzilla.redhat.com/1800597 Fixes https://bugs.gnu.org/39485 2020-02-09 Pádraig Brady maint: avoid syntax-check failure in previous commit * configure.ac: Restrict lines to 80 chars. 2020-02-09 Jim Meyering build: suppress new FP warning from gcc-10.0.1 * configure.ac (GNULIB_WARN_CFLAGS): Add -Wno-return-local-addr to avoid FP warning about careadlinkat.c. Discussed starting in https://lists.gnu.org/r/coreutils/2020-02/msg00006.html 2020-02-04 Pádraig Brady build: update to latest gnulib Pick up recent build fixes to avoid sysctl.h inclusion on glibc systems, restrict the max file size supported by read-file to PTRDIFF_MAX, and to avoid a -Werror=unused failure in test-canonicalize. tests: avoid false failure due to varying /proc/kallsyms * tests/cp/proc-short-read.sh: Switch to using /proc/cpuinfo, rather than /proc/kallsyms which was seen to vary in some cases. Fixes https://bugs.gnu.org/39357 2020-02-04 Pádraig Brady rmdir: fix --ignore-fail-on-non-empty with permissions errors Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty` had reversed the failure status for directories that failed to be removed for permissions reasons. I.E. it would have returned a failure status for such non empty dirs, and vice versa. * src/rmdir.c (errno_may_be_non_empty): Rename from the more confusing errno_may_be_empty(), and remove the EEXIST case (specific to Solaris), which is moot here since handled in errno_rmdir_non_empty(). (ignorable_failure): Fix the logic error so that _non_ empty dirs are deemed to have ignorable failures. (main): Fix clobbering of errno by is_empty_dir(). (remove_parents): Likewise. * tests/rmdir/ignore.sh: Add a test case. * THANKS.in: Add reporter who fixed the errno handling. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/39364 2020-02-03 Pádraig Brady build: avoid vector performance warnings in randperm * configure.ac: Add -Wno-vector-operation-performance to suppress the following gcc-9.2 error in gl/lib/randperm.c: error: vector operation will be expanded piecewise build: avoid including sysctl.h on glibc * src/uname.c: Avoid unneeded header with GLIBC, which has been deprecated since glibc-2.30. * src/uptime.c: Likewise. ls: support --time=creation to show/sort birth time * src/ls.c (usage): Reorganize help for --time, and add description for --time=birth. (do_statx): Store btime in mtime if available. (get_stat_btime): A new function to read the creation time from the appropriate stat structure member. (cmp_btime): A new function to compare birth time. (print_long_format): Output '?' when birth time unavailable. * doc/coreutils.texi: Document --time={birth,creation}. * tests/local.mk: Reference the new test. * tests/ls/birthtime.sh: Add a new test. * NEWS: Mention the new feature. 2020-01-30 Chris Meyering build: rearrange yes(1) code to prevent GCC 10 warning * src/yes.c (main): Convert for loop to do-while in order to indicate that the loop will be run at least once. This avoids the following warning after the second loop: src/yes.c:110:20: error: writing 1 byte into a region of size 0 2020-01-01 Emil Engler maint: add lib/iconv_open-zos.h to .gitignore * .gitignore: Add file newly generated by gnulib commit 49e78fc 2020-01-01 Pádraig Brady build: auto enable use of openssl with >= version 3 * configure.ac: Set --with-openssl=auto-gpl-compat as the default, so that openssl is used for md5sum etc., with openssl >= 3, which is newly licensed under ASL v2. * gnulib: Update to include "auto-gpl-compat" support. maint: adjust to split out xstrtol-error gnulib module * bootstrap.conf: Depend on the new module split from xstrtol. * src/df.c: Include "xstrtol-error.h" for xstrtol_fatal. * src/du.c: Likewise. * src/ls.c: Likewise. * src/od.c: Likewise. * src/pr.c: Likewise. * src/sort.c: Likewise. 2020-01-01 Pádraig Brady maint: update all copyright year number ranges Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year. 2019-12-08 Bernhard Voelker doc: add example to demonstrate sub-second sleep times * doc/coreutils.texi (sleep invocation): Add an example to demonstrate how to use the floating-point and the scientific notation to sleep for sub-second times, e.g. milli-, micro- and nanoseconds. Inspired by Stephane Chazelas in: https://lists.gnu.org/r/coreutils/2019-12/msg00005.html 2019-12-02 Pádraig Brady maint: fix syntax-check failure from recent adjustment * cfg.mk (old_NEWS_hash): Regenerate after commit v8.31-56-gc1e1965. 2019-12-02 Kamil Dudka chcon: do not validate security context if SELinux is disabled * src/chcon.c (main): Skip call of security_check_context() in case SELinux is disabled to avoid unnecessary failure. Bug: https://bugzilla.redhat.com/1777831 2019-11-12 Paul Eggert doc: remove colon from node name * doc/sort-version.texi (Minus/Hyphen and Colon characters): Rename from “Minus/Hyphen @samp{-} and Colon @samp{:} characters”, as texi2any 6.6 complains about colons in node names. shred: modernize documentation * doc/coreutils.texi (shred invocation): Modernize discussion to today’s technology (Bug#38168). * src/shred.c (usage): Omit lengthy duplication of the manual’s discussion of file systems and storage devices, as that became out of sync with the manual. Instead, just cite the manual. 2019-10-22 Paul Eggert all: improve parsing of numeric arguments This addresses a longstanding "update all callers" FIXME in lib/xstrtol.c, by having programs check that numbers do not have unknown suffixes. The problem was also reported for 'shuf' by my student Maggie Huang while reimplementing a shuf subset in Python as an exercise in UCLA Computer Science 35L: https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html This patch also improves the portability of the code to unusual platforms where ULONG_MAX < SIZE_MAX. * NEWS: Mention user-visible changes. * src/chgrp.c (parse_group): * src/chroot.c (parse_additional_groups): * src/du.c (main): * src/install.c (get_ids): * src/join.c (string_to_join_field): * src/ls.c (decode_switches): * src/md5sum.c (split_3): * src/shuf.c (main): * src/sort.c (specify_nthreads): * src/uniq.c (size_opt, main): Use uintmax_t instead of unsigned long, for portability to oddball platforms where unsigned long is not wide enough. * src/du.c (main): * src/expr.c (mpz_init_set_str) [!HAVE_GMP]: * src/install.c (get_ids): * src/ls.c (decode_switches): * src/mknod.c (main): * src/ptx.c (main): * src/shuf.c (main): * src/sort.c (specify_nmerge, specify_nthreads): Reject numbers with suffixes. * src/md5sum.c (split_3): Simplify. stdbuf: improve size checking * bootstrap.conf (gnulib_modules): Add minmax. * src/libstdbuf.c: Include stdint.h, minmax.h. (apply_mode): Don’t assume SIZE_MAX <= ULONG_MAX. Improve checking for invalid sizes. shuf: improve randperm overflow checking * gl/lib/randperm.c: Include randperm.h first, since it’s the API. Include stdint.h, count-leading-zeros.h, verify.h. (floor_lg): Rename from ceil_log (which was not actually implementing the ceiling!) and implement the floor using count_leading_zeros. (randperm_bound): Use floor_lg, not ceil_log. Use uintmax_t instead of size_t in case the size gets large on a 32-bit host. * gl/modules/randperm (Depends-on): Add count-leading-zeros, stdint. build: don’t worry about logical-op checking * configure.ac: Remove code tailoring --enable-gcc-warnings to GCC 4.7 and earlier, as developers no longer need to worry about GCCs that old. build: re-enable type-limits checking * configure.ac: When --enable-gcc-warnings is used, omit -Wno-type-limits. The need for -Wno-type-limits has passed, now that intprops.h uses builtin primitives for GCC 5 and later, given that recent GCCs issue type-limits warnings only for non-constant expressions. --enable-gcc-warnings is not intended for use with old compilers, so we can drop -Wno-type-limits now. 2019-10-21 Paul Eggert shuf: fix bug with ‘-r -n 0’ ‘shuf -r -n 0 file’ would mistakenly read from standard input. Problem reported by my student Jingnong Qu while reimplementing a shuf subset in Python as an exercise in UCLA Computer Science 35L: https://web.cs.ucla.edu/classes/fall19/cs35L/assign/assign3.html * NEWS: Mention the fix. Also, ASCIIfy a previous item. * src/shuf.c (main): Fix bug. * tests/misc/shuf.sh: Add a test case for the bug. 2019-10-09 Jeff Layton ls: use statx instead of stat when available statx allows ls to indicate interest in only certain inode metadata. This is potentially a win on networked/clustered/distributed file systems. In cases where we'd have to do a full, heavyweight stat() call we can now do a much lighter statx() call. As a real-world example, consider a file system like CephFS where one client is actively writing to a file and another client does an ls --color in the same directory. --color means that we need to fetch the mode of the file. Doing that with a stat() call means that we have to fetch the size and mtime in addition to the mode. The MDS in that situation will have to revoke caps in order to ensure that it has up-to-date values to report, which disrupts the writer. This has a measurable affect on performance. I ran a fio sequential write test on one cephfs client and had a second client do "ls --color" in a tight loop on the directory that held the file: Baseline -- no activity on the second client: WRITE: bw=76.7MiB/s (80.4MB/s), 76.7MiB/s-76.7MiB/s (80.4MB/s-80.4MB/s), io=4600MiB (4824MB), run=60016-60016msec Without this patch series, we see a noticable performance hit: WRITE: bw=70.4MiB/s (73.9MB/s), 70.4MiB/s-70.4MiB/s (73.9MB/s-73.9MB/s), io=4228MiB (4433MB), run=60012-60012msec With this patch series, we gain most of that ground back: WRITE: bw=75.9MiB/s (79.6MB/s), 75.9MiB/s-75.9MiB/s (79.6MB/s-79.6MB/s), io=4555MiB (4776MB), run=60019-60019msec * src/stat.c: move statx to stat struct conversion to new header... * src/statx.h: ...here. * src/ls.c: Add wrapper functions for stat/lstat/fstat calls, and add variants for when we are only interested in specific info. Add statx-enabled functions and set the request mask based on the output format and what values are needed. * NEWS: Mention the Improvement. 2019-10-03 Paul Eggert truncate: avoid integer-overflow assumptions * src/truncate.c (do_ftruncate): Simplify overflow checking, and don’t rely on theoretically-nonportable assumptions like assuming that OFF_MAX < UINTMAX_MAX. numfmt: avoid unlikely integer overflow * src/numfmt.c (parse_format_string): Report overflow if pad < -LONG_MAX, since that can’t be negated. nl: fix integer-overflow bug Problem reported by Roland Illig (Bug#37585) * src/nl.c (print_lineno): Don’t rely on undefined behavior when checking for integer overflow. cp: simplify integer overflow checking * src/copy.c (sparse_copy): Use INT_ADD_WRAPV instead of doing overflow checking by hand. 2019-09-08 Pádraig Brady seq: use faster processing for integer steps from 2 to 200 * src/seq.c: (seq_fast): Accept STEP as a parameter and use that to skip the output of generated numbers. (main): Relax to using seq_fast for integer steps between 1 and 200. For larger steps the throughput was faster using the standard incrementing procedure. (cmp): Use the equivalent but faster memcmp for equal len strings. * tests/misc/seq.pl: Update fast path cases. Addresses https://bugs.gnu.org/37241 2019-09-08 Pádraig Brady maint: use consistent header ordering and spacing in NEWS * NEWS: Move "Changes in behavior" before "New features", and ensure there is only a single blank line between sections. 2019-08-15 Paul Eggert build: update gnulib submodule to latest 2019-08-15 Assaf Gordon scripts: document how to build older versions on newer systems Based on https://lists.gnu.org/r/coreutils/2019-08/msg00011.html . * scripts/build-older-versions/README.older-versions: Documentation * scripts/build-older-versions/build-older-versions.sh: Helper script. * scripts/build-older-versions/.gitignore: Ignore build directory. * scripts/build-older-versions/coreutils-5.0-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-5.97-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-6.10-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-6.11-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-6.12-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-7.2-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-8.13-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-8.17-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-8.18-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-8.24-on-glibc-2.28.diff, scripts/build-older-versions/coreutils-8.4-on-glibc-2.28.diff: Patches. 2019-08-12 Bruno Haible build: adjust for recent gnulib pthread changes Discussed in https://lists.gnu.org/r/coreutils/2019-08/msg00030.html . * bootstrap.conf (gnulib_modules): Replace 'pthread' with pthread-* modules. * src/sort.c: Remove GNULIB_defined_pthread_functions conditional. 2019-08-11 Assaf Gordon date: mention military timezone changes from gnulib Gnulib commits f1f10d47be8762e4ca17c8957a0520b08d28abfb and 0673d8ab42c9bb0cf618a21b537cdd8fb976fb73 negated the meaning of military timezones parsed in gnu date. See https://lists.gnu.org/r/bug-gnulib/2019-08/msg00005.html and https://lists.gnu.org/r/coreutils/2019-08/msg00021.html * NEWS: Mention this user-visible change. * tests/misc/date.pl: Add tests for the new behavior. 2019-08-11 Bernhard Voelker maint: add lib/argmatch.h to po/POTFILES.in * po/POTFILES.in (lib/argmatch.h): Add to avoid sc_po_check error: "maint.mk: you have changed the set of files with translatable \ diagnostics;" 2019-08-11 Assaf Gordon gnulib: update to latest 2019-08-08 Pádraig Brady doc: clarify that truncate creates sparse files * src/truncate.c (usage): Explicitly mention "sparse". * doc/coreutils.texi (truncate invocation): Likewise. Addresses https://bugs.gnu.org/36963 2019-08-07 Mike Swanson dircolors: recognize the WebP image format * src/dircolors.hin: Add .webp for the WebP image format. Fixes https://bugs.gnu.org/36899 2019-08-07 Bernhard Voelker maint: fix error in syntax-check checking The previous commit introduced a bug into the following syntax-check, and thus effectively turned it off: $ make sc_prohibit_test_calls_print_ver_with_irrelevant_argument; \ echo $? prohibit_test_calls_print_ver_with_irrelevant_argument fatal: cannot change to 'grep': No such file or directory 0 * cfg.mk (sc_prohibit_test_calls_print_ver_with_irrelevant_argument): Remove changing directory, and pass $(srcdir) as argument to 'git -C'. 2019-08-04 Akim Demaille maint: fix issues in syntax-check * cfg.mk (sc_prohibit_colon_redirection): Don't expect `|` to denote the pipe character in git grep. (sc_tests_executable) (sc_case_insensitive_file_names) (sc_some_programs_must_avoid_exit_failure) (sc_prohibit_test_background_without_cleanup_) (sc_prohibit_test_calls_print_ver_with_irrelevant_argument) (sc_prohibit_test_ulimit_without_require_) (sc_prohibit_test_background_without_cleanup_) (sc_THANKS_in_duplicates) *sc_prohibit_test_calls_print_ver_with_irrelevant_argument): Don't expect builddir to be a descendant of srcdir. (sc_strftime_check): Don't check file size against 0 when "N\nq\n" was already put in the file. * THANKS.in: Remove me. 2019-08-03 Assaf Gordon seq: fix superfluous output line Under certain circumstances seq prints an extra line when the output format has custom format with characters following the printed numbers: $ seq -f "%g " 1000000 1000000 1e+06 1e+06 This is due to the "print_extra_number" logic using strings to determine whether a 'extra number' is needed, but only one string was trimmed when using a custom printf format. Prompted by https://lists.gnu.org/r/coreutils/2019-08/msg00001.html * NEWS: Mention fix. * src/seq.c (print_numbers): Trim the 'x0_str' string before comparing it to the previous 'x_str' string. * tests/misc/seq-extra-number.sh: Add this scenario. * tests/local.mk (all_tests): Add new test. 2019-07-22 Bernhard Voelker doc: improve new version sort chapter * doc/sort-version.texi: Fix some typos, avoid overly long lines in the generated PDF, enclose some sample strings in @samp{...} for better readability, etc. This also avoids an sc-avoid-builtin error: s/builtin/built-in/ 2019-07-15 Assaf Gordon doc: add "version sort ordering" chapter * doc/sort-version.texi: New file. * doc/local.mk (doc_coreutils_TEXINFOS): Add new file. * doc/coreutils.texi: @include new file, replace previous "Details about version sort" section. 2019-07-12 Andreas Dilger stat: don't explicitly request file size for filenames When calling 'stat -c %N' to print the filename, don't explicitly request the size of the file via statx(), as it may add overhead on some filesystems. The size is only needed to optimize an allocation for the relatively rare case of reading a symlink name, and the worst effect is a somewhat-too-large temporary buffer may be allocated for areadlink_with_size(), or internal retries if buffer is too small. The file size will be returned by statx() on most filesystems, even if not requested, unless the filesystem considers this to be too expensive for that file, in which case the tradeoff is worthwhile. * src/stat.c: Don't explicitly request STATX_SIZE for filenames. 2019-06-20 Paul Eggert od: use fseek on non-regular files Problem reported by Szőts Ákos (Bug#36291). * NEWS: Mention this. * src/od.c (skip): Try fseek even on files that do not have usable sizes, falling back on fread if fseek fails. 2019-06-18 Paul Eggert doc: mention ls -l user/group justification * doc/coreutils.texi (What information is listed): Document justification of user and group columns in ls -l output (Bug#36220). 2019-06-14 Jeff Layton stat: fix enabling of statx logic * src/stat.c: STATX_INO isn't defined until stat.h is included. Move the test down so it works properly. 2019-06-13 Assaf Gordon tests: avoid false-positive in date-debug test When debugging an invalid date due to DST switching, the intermediate 'normalized time' should not be checked - its value can differ between systems (e.g. glibc vs musl). Reported by Niklas Hambüchen in https://lists.gnu.org/r/coreutils/2019-05/msg00031.html Analyzed by Rich Felker in https://lists.gnu.org/r/coreutils/2019-05/msg00039.html * tests/misc/date-debug.sh: Replace the exact normalized time with 'XX:XX:XX' so different values would not trigger test failure. 2019-06-10 Jeff Layton stat: Use statx where available and support --cached * src/stat.c: Drop statbuf argument from out_epoch_sec(). Use statx() rather than [lf]stat() where available, so a separate call is not required to get birth time. Set STATX_* mask bits only for things we want to print, which can be more efficient on some file systems. Add a new --cache= command-line option that sets the appropriate hint flags in the statx call. These are primarily used with network file systems to indicate what level of cache coherency is desired. The new option is available unconditionally for better portability, and ignored where not implemented. * doc/coreutils.texi: Add documention for --cached. * man/stat.x (SEE ALSO): Mention statx(). * NEWS: Mention the new feature. 2019-06-09 Pádraig Brady doc: fix description of tail -f on truncated files * doc/coreutils.texi (tail invocation): Update to match the new behavior following commit v8.23-189-gb28ff6a 2019-06-08 Pádraig Brady split: fix failure for certain number of specified files * src/split.c (set_suffix_length): Use a more standard zero based logN calculation for the number of units. * tests/split/suffix-auto-length.sh: Add a test case. * THANKS.in: Mention the reporter. * NEWS: Mention the fix. Fixes https://bugs.gnu.org/35291 2019-05-30 Paul Eggert dd: be more careful about signal handling Problem reported by Hans Henrik Bergan (Bug#36007). * NEWS: Mention this. * src/dd.c (iclose, ifdatasync, ifstat, ifsync): New functions, which are more careful about SIGINT. (cleanup): Use iclose instead of close. (finish_up): Process signals first. (skip, dd_copy, main): Use ifstat instead of fstat. (dd_copy): Use ifdatasync and ifsync instead of fdatasync and fsync. maint: fix version number in NEWS 2019-05-28 Paul Eggert cp: fix /dev/stdin problem on Solaris Problem reported by Jakub Kulik (Bug#35713). * NEWS: Mention this. * configure.ac (DEV_FD_MIGHT_BE_CHR): New macro. * src/copy.c (DEV_FD_MIGHT_BE_CHR): Default to false. (follow_fstatat): New function. (copy_internal): Use it. * src/copy.h (XSTAT): Remove; no longer used. 2019-05-26 Kevin Locke doc: clarify dd sparse detection is by *output* block The wording of the dd --help text suggests that output will be skipped for sparse *input* blocks (i.e. that NUL-checking is done on input blocks) while the code actually checks/skips all-NUL *output* blocks.[1] * src/dd.c (usage): Update the --help text to clarify the above. * tests/dd/sparse.sh: Ensure sparseness is controlled with obs. [1]: https://superuser.com/a/1136358 2019-05-22 Martin Castillo doc: fix typo in sort set operations example * doc/coreutils.texi (sort invocation): Add a missing -u option to uniq. Addresses https://bugs.gnu.org/35849 2019-05-17 Paul Eggert b2sum: port blake2b-ref.c to HP-UX aCC Continue the fix for Bug#35650. * src/blake2/blake2b-ref.c [HAVE_CONFIG_H]: Include . 2019-05-15 Paul Eggert b2sum: sync better with upstream * src/blake2/b2sum.c: Reorder source code to minimize diffs from: https://github.com/BLAKE2/BLAKE2/blob/master/b2sum/b2sum.c b2sum: port to HP-UX aCC Its support for the -include option is flaky. Problem reported by Michael Osipov (Bug#35650). Plus, we could run into other compilers that don’t support any option like -include. Change the code so that -include is not needed. Although this causes us to depart from the upstream version, we’re already doing that for other reasons. * configure.ac (USE_XLC_INCLUDE): Remove, as there’s no guarantee a compiler will support something like -include. * src/blake2/b2sum.c [HAVE_CONFIG_H]: Include . * src/local.mk (src_b2sum_CPPFLAGS): Add -DHAVE_CONFIG_H. Do not use -include or a substitute. 2019-05-14 Paul Eggert stdbuf: port configure-time checking to HP-UX aCC Problem reported by Michael Osipov (Bug#35650). * configure.ac: Use AC_LANG_WERROR to pay attention to compiler and linker warnings when testing whether stdbuf will work. 2019-05-11 Paul Eggert b2sum: port to HP-UX C * src/blake2/blake2.h (BLAKE2_PACKED): Don’t assume __attribute__ ((packed)) works on non-Microsoft compilers. Instead, assume it works only if we have good reason to assume so, and fall back on Microsoft (or not packing) otherwise. In practice, not packing is good enough and the BLAKE2_PACKED macro is mostly just for documentation. cp: port fiemap.h to C99 * src/extent-scan.c (extent_scan_read): Adjust to change in struct fiemap. * src/fiemap.h (struct fiemap): Use FLEXIBLE_ARRAY_MEMBER to port to C99. basenc: port to C99 * src/basenc.c: Various minor style cleanups. (struct base_decode_context): Do not use anonymous unions, as they’re not in C99. Use a named union instead. All uses changed. maint: adjust to recent verify_true removal * src/system.h (X2NREALLOC, X2REALLOC, DECIMAL_DIGIT_ACCUMULATE): Use verify_expr instead of verify_true, which has been removed. (DECIMAL_DIGIT_ACCUMULATE): Remove unnecessary size check. build: update gnulib submodule to latest 2019-04-19 Bernhard Voelker gnulib: update to the latest * gnulib: Update to latest, mainly for: > mountlist: make parsing /proc/self/mountinfo more robust * NEWS: Mention the fix. Fixes https://bugs.gnu.org/33468 2019-03-31 Shugo Maeda factor: output immediately if stdout is a tty but stdin is not * src/factor.c (lbuf_putc): Use line buffered mode if the standard output is a terminal in the same way as the stdio library. User programs might use pty only for the standard out like the example of Ruby's PTY module: https://docs.ruby-lang.org/en/2.6.0/PTY.html#module-PTY-label-Example * NEWS: Mention the fix. Fixes https://bugs.gnu.orv/35046 2019-03-30 Pádraig Brady maint: fix syntax check failure * src/ln.c: Remove leading TAB. 2019-03-30 Martin Castillo maint: tee: use STDIN_FILENO rather than 0 * src/tee.c (tee_files): Use the name rather than the value. Addresses https://bugs.gnu.org/35041 2019-03-20 Paul Eggert dd: improve doc of stderr output * doc/coreutils.texi (dd invocation): Document stderr output more carefully. Say that conv=block can lose input data. 2019-03-18 Kamil Dudka md5sum,b2sum,sha*sum: --help: add note about binary/text mode * src/md5sum.c (usage): Make it clear that there is no difference between binary mode and text mode on GNU systems. Bug: https://bugzilla.redhat.com/406981 Bug: https://bugzilla.redhat.com/1688740 2019-03-17 Paul Eggert doc: add NEWS item for Solaris symlink fix ln: port to symlink ("x", ".") failing with EINVAL Problem reported by John Marino (Bug#34894). * src/ln.c (main): Port ln -s to Solaris symlink function, where symlink ("x", ".") fails with errno == EINVAL. 2019-03-16 Pádraig Brady doc: add a NEWS entry for the ln O_DIRECTORY fix * NEWS: Mention the bugfix. 2019-03-16 Paul Eggert ln: port to platforms lacking O_DIRECTORY * src/ln.c (main): Port to older platforms lacking support for POSIX.1-2008’s O_DIRECTORY flag (Bug#34876). 2019-03-15 Kamil Dudka doc: improve wording of the --kibibytes option description Bug: https://bugzilla.redhat.com/1527391 , https://bugs.gnu.org/33646 * doc/coreutils.texi (General output formatting): Improve wording of '--kibibytes' option. 2019-03-11 Bernhard Voelker maint: sync extra files from gnulib Some files are physically copied from gnulib, and should get sync'ed after each update to latest gnulib. This was forgotten during recent updates. * COPYING: Merge from gnulib/doc/COPYINGv3. * tests/init.sh: Merge from gnulib/tests/init.sh. 2019-03-11 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. 2019-03-10 Pádraig Brady version 8.31 * NEWS: Record release date. tests: test-N: include subsecond values in gating check * tests/misc/test-N.sh: The subsecond values for atime and mtime were potentially seen to differ on newlyl created files. So we include the subsecond portion when comparing stat values. tests: wc-nbsp: fix false failures on various systems * tests/misc/wc-nbsp.sh: Add gating checks for all characters, as there are disparate classifications on various systems: SunOS 5.10 treats \u202F, \u2060 as !iswprint() SunOS 5.10 treats \u00A0, \u2007 as iswspace() AIX 7.2, Darwin 17.4.0, NetBSD 7.1 treat \u2060 as !iswprint() 2019-03-07 Pádraig Brady tests: tail-2/pipe-f: avoid false failure closing stdout * tests/tail-2/pipe-f.sh: Check closing stdout with >&- is effective, which avoids a false failure on NetBSD 7.1 Reported by Assaf Gordon tests: tac-2-nonseekable: ensure we don't block indefinitely * tests/misc/tac-2-nonseekable.sh: Add a timeout to both protect and check whether we can close stdin correctly. tests: id/zero: avoid false failure due to sed differences * tests/id/zero.sh: sed on OSX will output a \n even if the input doesn't have a \n on the last "line". So ensure we always have a trailing '\n' to avoid the disparity. 2019-03-07 Pádraig Brady tests: test-N: fix false positives on some systems Testing by Assaf Gordon on OSX showed the atime wasn't being updated when explicitly set back in time. Also Debian 8.11 / mips64 was seen to not update the mtime when truncating an empty file. * tests/misc/test-N.sh: Isolate from different timestamping behaviors of various (file) systems, by correlating the timestamps with stat(1) before using `test -N`. 2019-03-07 Assaf Gordon doc: replace @hashchar{} with actual hash character Very old makeinfo-4.13 fails with: ./doc/coreutils.texi:2286: Unknown command `hashchar'. ./doc/coreutils.texi:2286: Misplaced {. ./doc/coreutils.texi:2286: Misplaced }. Reported Bernhard Voelker in https://lists.gnu.org/r/coreutils/2019-03/msg00016.html . * doc/coreutils.texi (basenc invocation): Replace @hashchar{} with actual hash character. The special syntax is only required when referring to #line directives. 2019-03-06 Pádraig Brady build: avoid statx related build failure on AIX * src/stat.c (get_birthtime): Check also for STATX_BTIME define, as a different statx is available on AIX 7.2. tests: wc-nbsp.sh: avoid failure on FreeBSD * tests/misc/wc-nbsp.sh: FreeBSD and OS X don't treat non breaking space as printable characters. So use wc -L to determine printability before testing non breaking space functionality. build: fix env build where SIGNUM_BOUND is not constant * src/env.c (initialize_signals): A new function to initialize the signals array on the heap, to avoid a build failure on opensolaris, where SIGNUM_BOUND is not a constant. 2019-03-04 Pádraig Brady doc: remove older ChangeLog items * Makefile.am: Update the oldest documented version to 8.22 which is now about 5 years old. build: revert recent change with distributed man page handling * man/local.mk: commit f114495e added an extra check to ensure a binary was working before using it to generate the man page. However this was not working for the false(1) command, and also one can generally specify that one should not be using generated commands on the current system by passing 'cross_compiling=yes' to the configure invocation. env: add --list-signal-handling to output non default handling * src/env.c (main): Output blocked or ignored signals before a command is executed. * doc/coreutils.texi (env invocation): Add the option. * tests/misc/env-signal-handler.sh: Add a test case. * NEWS: Mention the new feature. 2019-03-04 Assaf Gordon env: new options --{default,ignore,block}-signal[=SIG] New options to set signal handlers for the command being executed. --block-signal suggested by Paul Eggert in http://bugs.gnu.org/34488#71 --default-signal is useful to overcome the POSIX limitation that shell must not override inherited signal state, e.g. the second 'trap' here is a no-op: trap '' PIPE && sh -c 'trap - PIPE ; seq inf | head -n1' Instead use: trap '' PIPE && sh -c 'env --default-signal=PIPE seq inf | head -n1' Similarly, the following will prevent CTRL-C from terminating the program: env --ignore-signal=INT seq inf > /dev/null See https://bugs.gnu.org/34488#8 * NEWS: Mention new options. * doc/coreutils.texi (env invocation): Document new options. * man/env.x: Add example of --default-signal=SIG usage. (SEE ALSO): Mention sigprocmask. * src/env.c (signals): New global variable. (longopts): Add new options. (usage): Print new options. (parse_signal_params): Parse comma-separated list of signals, store in signals variable. (reset_signal_handlers): Set each signal to SIG_DFL/SIG_IGN. (parse_block_signal_params): Parse command-line options. (set_signal_proc_mask): Call sigprocmask to block/unblock signals. (main): Process new options. * src/local.mk (src_env_SOURCES): Add operand2sig.c. * tests/misc/env-signal-handler.sh: New test. * tests/local.mk (all_tests): Add new test. 2019-03-04 Martin Bukatovic stat: print birth time on systems supporting statx * configure.ac: Check for statx(), available on glibc >= 2.28. * src/stat.c (get_birthtime): Call statx() when available. * NEWS: Mention the improvement. 2019-03-04 Pádraig Brady df: support different file system encodings when not using tty * src/df.c (replace_problematic_chars): A new wrapper to be more conservative in our replacement when not connected to a tty. * tests/df/problematic-chars.sh: Add a test case. maint: tidy up recent additions to NEWS * NEWS: Move date change to improvements and fix nohup grammar. 2019-02-27 Bernhard Voelker doc: further clarify 'yes' alternative in seq invocation * doc/coreutils.texi (node seq invocation): Clarify to use the tool 'yes'; otherwise the reader may interpret the sentence as if one could pass 'yes' as the INCREMENT value. 2019-02-26 Pádraig Brady wc: treat non breaking space as a word separator * src/wc.c (iswnbspace): A new function to match characters in this class. (isnbspace): Likewise for single byte charsets. (main): Initialize posixly_correct from the environment, to allow disabling honoring NBSP in non C locales. (wc): Call is[w]nbspace() along with is[w]space. * bootstrap.conf: Ensure btowc is available. * tests/misc/wc-nbsp.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the change in behavior. 2019-02-25 Paul Eggert doc: more date +%F clarifications * doc/coreutils.texi (Date conversion specifiers): Plain %F is actually like %+4Y-%m-%d. (Padding and other flags): Mention POSIX restrictions. * src/date.c (usage): Document recent changes. doc: give date +%+F example * doc/coreutils.texi (Padding and other flags): Give example for + conversion specification. doc: fix typo in previous patch date: ‘+’ conversion specification flag The recent Gnulib update fixed Bug#34608; document and test this. * NEWS: Mention the change. * doc/coreutils.texi (Padding and other flags): Update doc to cover new flag and other POSIX.1-2017 changes. * tests/misc/date.pl (date-century-plus): New test. build: update gnulib submodule to latest 2019-02-24 Bernhard Voelker all: detect --help and --version more consistently For select programs which accept only --help and --version options (in addition to non-option arguments), process these options before any other options. Before: $ dd bs=1 --help dd: unrecognized option '--help' Try 'dd --help' for more information. $ yes me --help me --help me --help ... After: Any occurrence of '--help' in the arguments (prior to '--') will show the help screen. Discussed in https://bugs.gnu.org/33468 . * NEWS: Mention change. * src/cksum.c, src/dd.c, src/hostid.c, src/hostname.c, src/link.c, src/logname.c, src/nohup.c, src/sleep.c, src/tsort.c, src/unlink.c, src/uptime.c, src/users.c, src/whoami.c, src/yes.c (main): Replace parse_long_options() + getopt_long() calls with parse_gnu_standard_options_only(); Remove inclusion; Remove empty 'struct long_options' variable; * tests/misc/help-version-getopt.sh: Add test. * tests/local.mk (all_tests): Reference it. 2019-02-24 Pádraig Brady gnulib: update to the latest update to a version with parse_gnu_standard_options_only() 2019-02-20 Martin Castillo doc: fix join examples in texinfo * doc/coreutils.texi (join invocation): Fix various errors. Fixes https://bugs.gnu.org/34583 Fixes https://bugs.gnu.org/34584 2019-02-19 Daming Yang ls: better align month abbreviations containing digits * src/ls.c (abmon_init): Align numeric abbreviations right. * NEWS: Mention the improvement. 2019-02-18 Pádraig Brady sort: clarify in --debug; only text comparisons affected * src/sort.c (main): Adjust the debug info regarding locales, to clarify that only textual comparisons are affected. * tests/misc/sort-debug-warn.sh: Adjust accordingly. Fixes https://bugs.gnu.org/34490 2019-02-12 Pádraig Brady comm,join: ensure warnings are apparent upon exit * src/comm.c (main): Output a warning right before exit, in case previous errors have scrolled from view. * src/join.c (main): Likewise. * tests/misc/comm.pl: Addjust accordingly. * tests/misc/join.pl: Likewise. Fixes https://bugs.gnu.org/34347 2019-02-12 Filipp Gunbin doc: fix typo referencing RFC 2822 * doc/coreutils.texi (date invocation): s/822/2822/. Fixes https://bugs.gnu.org/34438 2019-02-11 Pádraig Brady gnulib: update to use new strtold module * gnulib: Update to make the new strtold module available. * bootstrap.conf: strtod is now a dependency of c-strtod, which in turn is a dependency of cl-strtod. This treats strtold and strtod similarly. * gl/lib/cl-strtod.c: Adjust to assume strtold is available. * tests/misc/sort-float.sh: Likewise. * src/sort.c: Likewise. (nan_compare): Adjust comment to indicate we still have to init padding bits as per https://sourceware.org/bugzilla/show_bug.cgi?id=13246 2019-02-04 Pádraig Brady seq: output decimal points consistently with invalid locales * src/seq.c (print_numbers): Only reset the locale if it was successfully set originally. * tests/misc/seq-locale.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. build: ensure sys/select.h is included bootstrap.conf: Explicitly depend on select, rather than transitively. * src/tail.c: Unconditionally include select.h as we use select() outside inotify contexts now. stat,tail: fix android build and support inotify * src/extract-magic: Treat android like linux, which fixes the build by ensuring the constants are defined. * src/stat.c: Support all constants on android, including the android specific "sdcardfs". * src/tail.c: Fix inclusion of statfs headers to be independent of inotify availability, as fremote() is used on linux even if inotify has been disabled. Also enable fremote() on android. * NEWS: Mention the improvment. Fixes https://bugs.gnu.org/34239 2019-01-31 Pádraig Brady tests: add test for locale decimal processing * tests/misc/sleep.sh: Check locale processing of printf, sleep, and timeout, when the french locale data is available. 2019-01-31 Pádraig Brady build: fix recent build failure on systems without strtold Recently introduced in commit v8.30-50-geb73e23 * gl/lib/cl-strtod.c: Fall back to strtod() on systems without strtold() (like we already do in sort). 2019-01-28 Pádraig Brady maint: fix new syntax-check failure from recent change * cfg.mk: Exclude cl-strtold.c wrapper from requiring config.h 2019-01-27 Paul Eggert printf,seq: remove c-strtod dependency * gl/modules/cl-strtold (Files): Add lib/cl-strtod.c, lib/cl-strtod.h. (Depends-on): Remove cl-strtod. (configure.ac): Redquire AC_C_RESTRICT. printf,seq: improve long double accuracy This fixes a thinko in the previous patch. * gl/lib/cl-strtod.c (STRTOD): New macro. (CL_STRTOD): Use it. printf,seq,sleep,tail,timeout: accept current-locale floats These commands now accept floating-point numbers in the current locale, as well as in the C locale. Compatibility problem reported by Robert Elz. * NEWS: Document this. * bootstrap.conf (gnulib_modules): Add cl-strtod, cl-strtold. Remove c-strtold. * doc/coreutils.texi (Floating point, tail invocation) (printf invocation, timeout invocation, sleep invocation) (seq invocation): Document this. * gl/lib/cl-strtod.c, gl/lib/cl-strtod.h, gl/lib/cl-strtold.c: * gl/modules/cl-strtod, gl/modules/cl-strtold: New files. * src/printf.c, src/seq.c, src/sleep.c, src/tail.c, src/timeout.c: Include cl-strtod.h instead of c-strtod. * src/printf.c (vstrtold): * src/seq.c (scan_arg, print_numbers): * src/sleep.c (main): * src/tail.c (parse_options): * src/timeout.c (parse_duration): Use cl_strtold instead of c_strtold. 2019-01-25 Paul Eggert doc: update Goldberg URL * doc/coreutils.texi (Floating point): Update URL. sleep: improve doc Problem reported by Robert Elz. * doc/coreutils.texi (sleep invocation): Say that arguments must be non-negative, which means they cannot be arbitrary floating-point numbers. Mention POSIX, not “historical implementations” that are no longer of practical interest. List the extensions to POSIX. * src/sleep.c (usage): Omit needless words, removing dubious commentary about “most implementations” and incorrect commentary about “arbitrary”. Details about exactly which numbers are allowed can be found in the documentation. 2019-01-20 Pádraig Brady tail: fix handling of broken pipes with SIGPIPE ignored * init.cfg (trap_sigpipe_or_skip_): A new function refactored from... * tests/misc/printf-surprise.sh: ...here. * tests/misc/seq-epipe.sh. Likewise. * src/tail.c (die_pipe): Ensure we exit upon sending SIGPIPE. * tests/tail-2/pipe-f.sh: Ensure we exit even if SIGPIPE is ignored. * NEWS: Mention the bug fix. 2019-01-20 Ayappan tail: fix recent ineffective AIX change * src/tail.c: Fix commit v8.30-40-gd5ab4cb which was ineffective. Fixes http://bugs.gnu.org/33946 2019-01-20 Pádraig Brady build: ensure VLAs are not used Fail developer builds if VLAs are used, as there are portability concerns to consider with them. * configure.ac: Enable -Wvla which is implicit in the full list added. * m4/jm-macros.m4: Define GNULIB_NO_VLA which disables use of VLAs within gnulib code. 2019-01-20 Pádraig Brady gnulib: update to the latest * gnulib: Update to a version supporting GNULIB_NO_VLA * bootstrap: Sync with latest 2019-01-16 Bernhard Voelker build: use distributed man pages when running with --help fails When building against an incompatible GLIBC version compared to that on the build host, then running the just-built binary might fail although it is the same platform - thus CROSS_COMPILING is false. As a result, generating the man pages fails. * man/local.mk (.x.1): Add a check to verify that running the utility with --help succeeds, otherwise falling back to using 'dummy-man'. 2019-01-13 Pádraig Brady ls: with --group-directories-first, also group symlinked dirs * src/ls.c (is_linked_directory): A new function to also consider symlinked directories. (main): Rename check_symlink_color to check_symlink_mode, and enable that with --group-directories-first. (DIRFIRST_CHECK): Adjust to use is_linked_directory, rather than just is_directory. (gobble_file): Simplify to always update f->linkmode if the stat() succeeds. * tests/ls/group-dirs.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the change in behavior. Suggested by Amin Bandali in https://lists.gnu.org/r/coreutils/2018-12/msg00017.html tail: don't exit immediately with filters on AIX * src/tail.c: Fix the check_output_available check on AIX. Note we don't use poll for all systems as the overhead of adding the gnulib poll module wouldn't be worth it just for this single use. * tests/tail-2/pipe-f.sh: Fix the test which always passed due to only the exit code of sleep being checked. * NEWS: Mention the bug fix and rearrange alphabetically. Fixes http://bugs.gnu.org/33946 2019-01-06 Assaf Gordon basenc: allocate buffers on heap Allocate the encoding/decoding buffers dynamically on the heap instead of using variable-length-array (VLA) on the stack. Discussed in https://lists.gnu.org/r/coreutils/2019-01/msg00004.html . * src/basenc.c (do_encode,do_decode): Allocate inbuf/outbuf using xmalloc, and free if using LINT. 2019-01-04 Pádraig Brady doc: adjust URLs in help to avoid wrapping * src/system.h: Adjust lines containing URLs so that they don't wrap on 80 column terminals. One could also use .UR macros, but these aren't universally available. Note the adjustments here need to be compatible with the pattern matching done in help2man. Addresses https://bugs.gnu.org/33914 2019-01-01 Assaf Gordon maint: update all copyright year number ranges Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year. 2019-01-01 Bernhard Voelker maint: mention base32 in the title line of common basenc.c * src/basenc.c: Do the above, and remove a redundant comment. 2019-01-01 Assaf Gordon base64,base32: fix 'extra operand' error message In the following invocation, 'a' is the input file, and 'b' is the extra operand: $ base64 a b Report 'b' in the error message instead of 'a': $ base64 a b base64: extra operand 'b' Discussed in https://lists.gnu.org/r/coreutils/2018-12/msg00008.html . * src/basenc.c (main): If there is more than one non-option operand, report the second one (assuming the first is a the input file name). * tests/misc/base64.pl: Add tests. * tests/misc/basenc.pl: Adjust expectedc error message in tests. * NEWS: Mention bugfix. 2018-12-31 Pádraig Brady doc: mention that more than 8 colors are supported by ls * src/dircolors.hin: Mention any codes supported by the terminal are allowed. Addresses https://bugs.gnu.org/33915 2018-12-28 Assaf Gordon basenc: A new program complementary to base64/base32 Encodes/decodes data in various common formats: base64,base64url,base32,base32,base16,base2,z85. Discussed here: https://lists.gnu.org/r/coreutils/2018-11/msg00014.html https://lists.gnu.org/r/coreutils/2018-12/msg00019.html * AUTHORS: Add basenc. * README: Reference the new program. * NEWS: Mention the new program. * build-aux/gen-lists-of-programs.sh: Add basenc. * doc/coreutils.texi: (basenc invocation): Document the new command. * man/.gitignore: Ignore the generated man page. * man/basenc.x: A new template, with few examples. * man/local.mk: Reference the new man page. * scripts/git-hooks/commit-msg: Allow basenc as program prefix. * src/.gitignore: Ignore the new binary. * src/basenc.c: (usage): Mention new options. (main): Handle new options. (isbase*, base*_length, base*_encode, base*_decode_ctx): Implement new encoding/decoding formats. * src/local.mk: Add new program. * tests/local.mk: Add new test. * tests/misc/basenc.pl: New tests. * tests/misc/help-version.sh (basenc_setup): use '--version' for default invocation (basenc errors with no parameters). 2018-12-21 Assaf Gordon maint: rename base64.c to basenc.c In preparation for adding 'basenc' program. Suggested in https://lists.gnu.org/r/coreutils/2018-11/msg00019.html . * src/base64.c: Rename to src/basenc.c. * src/local.mk: Update base*_SOURCES definitions. * po/POTFILEs.in: Rename base64 to basenc. 2018-12-15 Paul Eggert shred,sort,split: add NEWS item shred,sort,split: fix ftruncate error reporting Problem reported for split by Scott Worley (Bug#33761): * src/shred.c (do_wipefd): Also report an error if ftruncate fails on a shared memory object. * src/sort.c (get_outstatus): New function. (stream_open, avoid_trashing_input): Use it. * src/sort.c (stream_open): * src/split.c (create): If ftruncate fails, do not report an error unless it is a regular file or a shared memory object. 2018-11-07 Bernhard Voelker sync: add NEWS and test for the fix in the previous commit * NEWS (Bug fixes): Mention the fix in commit 94d364f157f0. While at it, remove duplicate "Changes in behavior" heading. * tests/misc/sync.sh: Add a test with a write-only file for the fix. 2018-11-06 Paul Eggert sync: fix open fallback bug Problem caught by Coverity Analysis and reported by Kamil Dudka (Bug#33287). * src/sync.c (sync_arg): Fix typo in fallback code. 2018-10-28 Paul Eggert ln: use linkat and symlinkat Open a target directory and use its file descriptor in linkat, symlinkat, etc. syscalls, instead of constructing long file names by concatenating the target directory name to a basename. This avoids O(N²) behavior with ‘ln F1 F2 ... Fn DIR’ when DIR is a long file name with many slashes. It also avoids some races if DIR is renamed while ln is running. * bootstrap.conf (gnulib_modules): Add openat-safer. * src/ln.c: Include fcntl-safer.h. (O_PATHSEARCH): New constant. (errno_nonexisting, target_directory_operand): Remove; no longer used. (atomic_link, do_link): New arg DESTDIR_FD. All uses changed. (do_link): New arg DEST_BASE. All uses changed. (main): Open target directory and use its file descriptor as DESTDIR_FD. build: update gnulib submodule to latest * src/copy.c (copy_internal): * src/cp.c (do_copy): * src/ln.c (do_link): Adjust to Gnulib API change. 2018-10-27 Bernhard Voelker tests: provide 100% coverage for echo * src/echo.c (usage): Assert that STATUS is always EXIT_SUCCESS. * tests/misc/echo.sh: Add further tests for all hex and escape and escape characters. To get coverage statistics, run: make coverage -j 4 TESTS=tests/misc/echo.sh SUBDIRS=. xdg-open doc/coverage/src/echo.c.gcov.frameset.html 2018-10-27 Pádraig Brady echo: always process escapes when POSIXLY_CORRECT is set * src/echo.c (main): Always enable backslash processing if POSIXLY_CORRECT is set. * tests/misc/echo.sh: Add (the first) test for the echo command. * tests/local.mk: Reference the new test. * tests/misc/printf.sh: Update a stale comment. * doc/coreutils.texi (echo invocation). Mention that POSIXLY_CORRECT now always enables backslash processing. * NEWS: Mention the change in behavior. Fixes https://bugs.gnu.org/32703 Issue identified by Eric Blake. 2018-10-26 Bernhard Voelker test: add -N unary operator Bash knows 'test -N FILE'. Add it to GNU 'test' as well. * src/test.c (unary_operator): Add a case for 'N'. (usage): Document it. * doc/coreutils.texi (node File characteristic tests): Likewise. * NEWS (New features): Likewise. * tests/misc/test-N.sh: Add a test. * tests/local.mk (all_tests): Reference it. 2018-10-26 Bernhard Voelker test: simplify redundant code Remove the function 'test_unop', as the cases therein are redundant to those handled by 'unary_operator'; exception: the cases 'o' and 'N': they had been present in test_unop and handling the commands test -N STR test -o STR and test x = x -a -N STR test x = x -a -o STR which ran into an error later on anyway. With this commit, the error diagnostic will change from ... $ /usr/bin/test -N STR /usr/bin/test: extra argument '-N' $ /usr/bin/test -o STR /usr/bin/test: extra argument '-o' ... to ... $ src/test -N STR src/test: '-N': unary operator expected $ src/test -o STR src/test: '-o': unary operator expected * src/test.c (test_unop): Remove. (unary_operator): Fail with test_syntax_error in the default case. (term): Directly call unary_operator. (two_arguments): Likewise. * tests/misc/test-diag.pl: Adjust error diagnostic. 2018-10-26 Bernhard Voelker test: remove support for the ambigous -a unary operator * src/test.c (unary_operator): Remove case 'a'. (test_unop): Likewise. * NEWS (Changes in behavior): Document the change. Discussed at https://bugs.gnu.org/33097 2018-10-21 Bernhard Voelker test: avoid FP in chroot-credentials.sh for different group list order On my openSUSE:Tumbleweed system, I get a false positive test failure in the above 'check-root' test because the group lists inside and outside the chroot have a different order: ++ chroot --userspec=berny / id -G ++ id -G berny + test '100 454 457 480 492' = '100 480 492 457 454' + fail=1 * tests/misc/chroot-credentials.sh (num_sort): Add function to sort group lists, and use it in the test cases which test multiple groups. 2018-10-20 Paul Eggert doc: tidy up setuid commentary * doc/perm.texi (Mode Structure): Improve wording. (Numeric Modes): Don’t say “on execution” (Bug#9594). 2018-10-19 Paul Eggert ln: avoid directory hard-link races Previously, 'ln A B' did 'stat("B"), lstat("A"), link("A","B")' where the stat and lstat were necessary to avoid hard-linking directories on systems that can hard-link directories. Now, in situations that prohibit hard links to directories, 'ln A B' merely does 'link("A","B")'. The new behavior avoids some races and should be more efficient. This patch was inspired by Bug#10020, which was about 'ln'. * bootstrap.conf (gnulib_modules): Add unlinkdir. * src/force-link.c (force_linkat, force_symlinkat): New arg for error number of previous try. Return error number, 0, or -1 if error, success, or success after removal. All callers changed. * src/ln.c: Include priv-set.h, unlinkdir.h. (beware_hard_dir_link): New static var. (errnoize, atomic_link): New functions. (target_directory_operand): Use errnoize for simplicity. (do_link): New arg for error number of previous try. All callers changed. Do each link atomically if possible. (main): Do -r check earlier. Remove linkdir privileges so we can use a single linkat/symlinkat instead of a racy substitute for the common case of 'ln A B' and 'ln -s A B'. Set beware_hard_dir_link to disable this optimization. cp: 'cp -il A B' no longer fails if user OKs it * NEWS: Mention the change. * src/copy.c (copy_internal): Replace the link if the user has okayed it. build: update gnulib submodule to latest * gl/modules/tempname.diff: Update to match Gnulib. 2018-10-17 Paul Eggert doc: add chmod examples Discussed in https://bugs.gnu.org/11043 . * doc/coreutils.texi (chmod invocation): Add examples. 2018-10-02 Bjarni Ingi Gislason doc: fix minor mistakes in "env.x" * man/env.x (OPTIONS): Fix a spelling mistake. Protect a period at the beginning of a line. 2018-09-30 Achilles Gaikwad id: support multiple specified users $ id root nobody uid=0(root) gid=0(root) groups=0(root) uid=99(nobody) gid=99(nobody) groups=99(nobody) * src/id.c (main): Make variables opt_zero, just_group_list, just_group, use_real, just_user global to be used in a new function. (print_stuff): New function that will print user and group information for the specified USER. When using -G option delimit each record with two NULs. Restructure the code in the file to have global variables followed by functions. * tests/id/zero.sh: Add test cases to check the usage of -z option with multiple users. * tests/id/uid.sh: Add a test case to ensure all users are queried in the presence of errors. * doc/coreutils.texi: Document the interface changes. * NEWS: Mention the new feature. 2018-09-25 Stéphane Campinas doc: csplit: clarify handling of regexps with negative offsets * doc/coreutils.texi (csplit invocation): Detail the behavior with regexp patterns and negative offsets, which differs from line number patterns, to avoid looping on the input. For example: $ seq 50 | csplit -s - /15/-5 /12/ csplit: ‘/12/’: match not found 2018-09-24 Pádraig Brady doc: csplit: clarify input may not be reproducible from output * doc/coreutils.texi (csplit invocation): Clarify that portions of the input may be skipped and thus the input may not be reproducible by just concatenating the output files. Fixes https://bugs.gnu.org/32317 2018-07-27 Paul Eggert df: omit redundant comparison Trivial inefficiency reported by Bruno Haible in: http://lists.gnu.org/r/bug-gnulib/2018-07/msg00109.html * src/df.c (hide_problematic_chars): Omit redundant test. df: tune slightly * src/df.c (get_header, get_dev): Avoid calling mbswidth twice when once will do. df: avoid multibyte character corruption on macOS This improves on the earlier fix for the problem reported by Chih-Hsuan Yen (Bug#32236), by also looking for other control characters and for encoding errors. * src/df.c: Include wchar.h and wctype.h instead of c-ctype.h. (hide_problematic_chars): Process the string as multibyte. Use iswcntrl, not c_iscntrl. 2018-07-26 Chih-Hsuan Yen df: avoid multibyte character corruption on macOS * src/df.c (hide_problematic_chars): Use c_iscntrl() as passing 8 bit characters to iscntrl() is not supported on macOS. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/32236 2018-07-22 Wodry (tiny change) doc: improve documentation of binary prefixes * doc/coreutils.texi (Common options): * src/dd.c, src/head.c, src/od.c, src/stdbuf.c, src/tail.c (usage): * src/system.h (emit_size_note): Mention binary prefixes (Bug#32242). 2018-07-21 Pádraig Brady tests: avoid false failure on sparc 32 bit * tests/rm/rm-readdir-fail.sh: Skip the test entirely on 32 bit, so we avoid conflating the 32bit and 64 bit types, as that triggers alignment issues (SIGBUS) on Gentoo sparc. Fixes https://bugs.gnu.org/29886 2018-07-05 Paul Eggert build: update gnulib submodule to latest * bootstrap.conf, src/copy.c, src/mv.c, src/shred.c: Adjust to renaming of renameat2 to renameatu. 2018-07-05 Pádraig Brady tests: fix skipping in some tests * tests/cp/cp-a-selinux.sh: Use 'skip_' rather than the probably undefined 'skip'. * tests/du/2g.sh: Likewise. * tests/install/install-Z-selinux.sh: Likewise. * tests/misc/chcon.sh: Likewise. * tests/misc/selinux.sh: Likewise. * tests/mkdir/restorecon.sh: Likewise. * cfg.mk (sc_prohibit-skip): A new syntax check to catch the issue. 2018-07-02 Jim Meyering maint: init.cfg: fix a minor test-related quoting bug * init.cfg (require_membership_in_two_groups_): This fixes a bug introduced by me in v8.15-8-gdd0e4c562. Luckily, the consequence of low-probability triggering the bug was the mere added backslash in the diagnostic: "...but running id -G\ either...". It would be triggered in a test failure for one who is a member of only one or fewer groups. 2018-07-02 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.30 * NEWS: Record release date. 2018-07-01 Pádraig Brady tests: standardize perl usage in tests * tests/cp/fiemap-FMR.sh: Ensure perl is parameterized to $PERL, and ensure require_perl_ is used, so tests are skipped appropriately. * tests/cp/preserve-gid.sh: Likewise. * tests/du/long-from-unreadable.sh: Likewise. * tests/misc/env-S-script.sh: Likewise. * tests/misc/sort-benchmark-random.sh: Likewise. * tests/rm/deep-2.sh: Likewise. maint: copy: avoid new static analyzer warnings * src/copy.c (copy_internal): Use the lint protected src_mode, rather than accessing the src_sb again. Also unconditionally populate src_sb when !x->move_mode and in lint mode. Reported by Kamil Dudka with coverity and clang analyzer. maint: fix recent stale comments and spelling mistakes * doc/coreutils.texi: s/seperator/separator/. * tests/misc/env-S.pl: Likewise. * src/env.c: Fix stale comment. 2018-06-27 Pádraig Brady maint: disable overly agressive sc_gitignore_redundant * cfg.mk (sc_gitignore_redundant): Disabled for now as too aggressive flagging entries like /lib/arg-nonnull.h in a newly checked out repo. env: adjust diagnostics provided for shebang usage * src/env.c (main): Don't process '-' specially since that causes an issue on the openbsd getopt implementation where a lone '-' is now processed as an option, and anyway it doesn't particuarly help diagnosing common shebang usage issues. Also don't restrict the extra diagnostics for shebang usage to the case with 3 arguments, as further arguments can be passed to a script. * tests/misc/env-S.pl: Adjust accordingly. 2018-06-27 Assaf Gordon tests: accept getopt errors without single-quotes On OpenBSD 6.2, invalid single options produce error messages without single quotes: $ ./src/chroot -/ chroot: unknown option -- / As opposed to other systems: ./src/chroot: invalid option -- '/' Modify the grep search to accept this. * tests/misc/usage_vs_getopt.sh (checkprg): Change the grep pattern to accomodate no-single-quotes cases. 2018-06-27 Pádraig Brady tests: fix false failures when perl not available * tests/local.mk: Reference the stub that skips perl tests, with the correct path. tests: fix false failure with limited shebang lines * tests/misc/env-S-script.sh: Provide a wrapper to emulate shebang processing, but without length limits, which are 127 on Linux for example. maint: update gnulib to latest * gnulib: Update to latest, which incorporates a thread linking fix from Bruno Haible, which was seen on newer Ubuntu systems. 2018-06-27 Assaf Gordon tests: remove unused Data::Dumper perl module The module is not needed anymore (was used during development). Despite being a Perl core module, platforms like CentOS don't install it by default. Reported by Bruno Haible at https://lists.gnu.org/r/coreutils/2018-06/msg00093.html. * tests/misc/csplit-suppress-matched.pl: Remove Data::Dumper. 2018-06-25 Carlos Santos maint: fix -Werror=suggest-attribute=malloc in expr.c Add attribute 'malloc' to mpz_get_str to prevent the following on GCC 8.1.1 src/expr.c:117:1: error: function might be candidate for attribute 'malloc' if it is known to return normally [-Werror=suggest-attribute=malloc] mpz_get_str (char const *str, int base, mpz_t z) ^~~~~~~~~~~ cc1: all warnings being treated as errors * src/expr.c (mpz_get_str): Add _GL_ATTRIBUTE_MALLOC. 2018-06-25 Pádraig Brady maint: update gnulib to latest * gnulib: Update to latest. * .gitignore: Add new entries. * bootstrap.conf: Enable wchar-single, which will enable more efficient replacements of wcwidth and mbrtowc, as we indicate that the charset will no change between invocations of these functions. maint: sync longlong.h from gmp repo * src/longlong.h: Sync changes. No functional change. maint: avoid false positive in src/fs-magic-compare * src/local.mk (fs_normalize_perl_subst): `make src/fs-magic-compare` was reporting incorrectly that AFS was not being handled. Add a mapping to our KAFS identifier. * .gitignore: Add intermediate files from `make src/fs-magic-compare` 2018-06-23 Bernhard Voelker tests: initialize fail=0 to avoid "unary operator expected" errors With an uninitialized variable 'fail', the unquoted use like test $fail = 1 lead to the shell error "unary operator expected". The uninitialized 'fail' variable was a side effect of https://git.sv.gnu.org/cgit/gnulib.git/commit/?id=e91c0d4f9 which was pulled into coreutils-v8.26 with https://git.sv.gnu.org/cgit/coreutils.git/commit/?id=ef9650170 Coreutils test code relied and relies on 'fail' to be initialized, so initialize that variable here. * tests/local.mk (TESTS_ENVIRONMENT): Initialize fail=0. 2018-06-21 Jim Meyering maint: do not depend directly on gnulib's now-unused ftello module * bootstrap.conf (gnulib_modules): Remove ftello, since it is no longer used directly, since v8.9-11-geab97b307. 2018-06-21 Pádraig Brady tests: provide an option to relax the need for gdb * tests/rm/r-root.sh: gdb provides extra protection, but is not strictly necessary. So provide an option for maintainers to relax the requirements. rm: add --preserve-root=all to protect mounts * src/remove.c (rm_fts): With the --preserve-root=all extension, reject command line arguments that are mount points. * src/remove.h (rm_options): Add preserve_all_root to store config. * src/mv.c (rm_option_init): Init preserve_all_root to false. * src/rm.c (main): Init preserve_all_root as per option. (usage): Describe the new option. * src/remove.c (rm_fts): Lookup the parent device id, and reject the cli argument if a separate file system. * tests/rm/one-file-system.sh: Add a test case. * NEWS: Mention the new feature. 2018-06-21 Adam Borowski cp: add --reflink=never to force standard copy mode This mode is currently the default, but most if not all users of reflink-capable filesystems want --reflink=auto, which is often encapsulated into an alias. Adding --reflink=never allows overriding such an alias. * doc/coreutils.texi (cp invocation): Describe the new option. * src/cp.c: Support --reflink=never. * tests/cp/reflink-auto.sh: Add a test case. * NEWS: Mention the new feature. 2018-06-21 Assaf Gordon env: add -S/--split-string option Adopted from FreeBSD's env(1), useful for specifing multiple parameters on a shebang (#!) script line, e.g: #!/usr/bin/env -S perl -w -T Discussed in https://lists.gnu.org/r/coreutils/2018-04/msg00011.html * src/env.c (valid_escape_sequence,escape_char,scan_varname, extract_varname,validate_split_str,build_argv, parse_split_string): New functions. (main): Process new option and call parse_split_string. (usage): Mention new option. * tests/misc/env-S.pl: Test new option from the command line. * tests/misc/env-S-script.sh: Test new option from shebang scripts. * tests/local.mk (all_tests): Add new tests. * man/env.x (OPTIONS): Show a brief example of -S usage and point to the full documentation for more information. * doc/coreutils.texi (env invocation): Detail usage of -S/--split-string option. * NEWS: Mention new option. 2018-06-21 Assaf Gordon env: add -v/--debug option Prints verbose information about each step: $ env -v -uFOO -C /tmp BAR=BAZ date -u env: unset: FOO env: setenv: BAR=BAZ env: chdir: '/tmp' env: executing: date env: arg[0]= ‘date’ env: arg[1]= ‘-u’ Sun Apr 22 08:52:30 UTC 2018 Inspired by FreeBSD's env(1). * src/env.c (usage): Mention new option. (main): Print debug information if requested. * NEWS: Mention new option. * doc/coreutils.texi (env invocation): Mention -v/--debug. 2018-06-21 Assaf Gordon maint: refactor unsetenv call in env Keep unset envvars (-uFOO) in an array for later deletion, instead of reiterating over argv. Done in preparation for '-S string' feature. Related to '-u' discussion in https://lists.gnu.org/r/coreutils/2018-04/msg00013.html * src/env.c (append_unset_var,unset_envvars): New functions. (main): Use new functions. 2018-06-21 Kaxandra Labat ls: ignore case when coloring file extensions * src/ls.c (get_color_indicator): s/STREQ_LEN/c_strncasecmp/ * src/dircolors.hin: Remove a now redundant entry. * tests/ls/color-ext.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the change in behavior. 2018-06-21 Pádraig Brady md5sum,b2sum,sha*sum: support -z,--zero option * doc/coreutils.texi (md5sum invocation): Describe the new option, and how it's not supported by --check, and how it disables escaping. * src/md5sum.c (delim): A new global to parmeterize the out delimiter. (main): Don't enable file name escaping with -z, and output '\0'. * tests/misc/md5sum-newline.pl: Add a test case. * NEWS: Mention the new feature. 2018-06-21 Pádraig Brady wc: optimize processing of ASCII in multi byte locales ===== Benchmark setup (on GNU/Linux) ==== $ yes áááááááááááááááááááá | head -n100000 > mbc.txt $ yes 12345678901234567890 | head -n100000 > num.txt ===== Before ==== $ time src/wc -Lm < mbc.txt real 0m0.186s $ time src/wc -m < mbc.txt real 0m0.186s $ time src/wc -Lm < num.txt real 0m0.055s $ time src/wc -m < num.txt real 0m0.056s ==== After ==== $ time src/wc -Lm < mbc.txt real 0m0.196s $ time src/wc -m < mbc.txt real 0m0.173s $ time src/wc -Lm < num.txt real 0m0.031s $ time src/wc -m < num.txt real 0m0.028s * src/wc.c (wc): Only call wide variant functions like iswprint() and wcwidth() for non is_basic() characters. I.E. non ISO C "basic character set" characters. This is especially significant on OSX where wcwidth() is very expensive (about 10x in tests). * NEWS: Mention the improvement. Suggested by Eric Fischer. 2018-06-14 Paul Eggert doc: port test.1 to doclifter * man/test.x: Use \& instead of quoting (Bug#31803). doc: port man pages to doclifter Problem reported by Eric S. Raymond (Bug#31803). * man/test.x: Add SYNOPSIS section, since help2man understandably gets confused by the square brackets. * src/ln.c (usage): Omit parenthetical "(Nth form)" in usage, as it confuses doclifter. 2018-06-04 Pádraig Brady cp: preserve existing permissions with --no-preserve=mode This issue was introduced in commit v8.19-145-g24ebca6 * src/copy.c (copy_internal): With --no-preserve=mode, only reset permissions for newly created files. (copy_reg): Likewise. * NEWS: Mention the fix. * tests/cp/preserve-mode.sh: Add a test case. Fixes https://bugs.gnu.org/31675 2018-05-29 Pádraig Brady tests: fix periodic false failure in month alignment * tests/ls/abmon-align.sh: Base relative month adjustment from the middle of the month, to avoid failures due to months being repeated. Fixes https://bugs.gnu.org/31644 2018-05-26 Bjarni Ingi Gislason doc: formatting fixes in "du.x" and "rm.x" Avoid warnings from: groff -b -e -mandoc -T utf8 -rF0 -t -w w -z * man/du.x: Change ".BR" to ".B" if there is only one argument. Protect an end-of-sentence indicator (.?!) with '\&' if it does not mean an end of a sentence. Change '--' to '\-\-' if it indicates an option. * man/rm.x: Change '\=' to '='. 2018-05-18 Pádraig Brady cp: with --force; replace self referential symlinks * src/copy.c (copy_internal): Don't fail immediately upon getting ELOOP when running stat() on the destination, rather proceeding if -f specified, allowing the link to be removed. If the loop is not in the final component of the destination path, we still fail but at the subsequent unlink() stage. * doc/coreutils.texi (cp invocation): Adjust wording to say that --force doesn't work with dangling links, rather than all links that can't be traversed. * tests/cp/thru-dangling.sh: Add a test case. * NEWS: Mention the change in behavior. Discussed in https://bugs.gnu.org/31335 2018-05-15 Pádraig Brady cp: fix symlink checks when overwriting files Ensure this _does_ recreate the symlink Given "path1" and "path2" are on different devices. $ touch "path1/file" $ cd path2/; ln -s path1/file $ cp -dsf path1/file . Ensure this does _not_ overwrite file $ touch file $ ln -s file l1 $ cp -sf l1 file * src/copy.c (same_file_ok): Remove device ids from consideration, instead deferring to future EXDEV with --link or allowing the first case above to work. Also ensure that we do not exist this function too early, when the destination file is not a symlink, which protects against the second case. * tests/cp/cross-dev-symlink.sh: Add a test for the first case. * tests/cp/same-file.sh: Add a test for the second case above. * NEWS: Mention the bug fixes. * THANKS.in: Mention the reporters who also analyzed the code. Fixes https://bugs.gnu.org/31364 2018-05-15 Pádraig Brady cp: ensure --remove-destination doesn't traverse symlinks * src/cp.c (target_directory_operand): Allow through inaccessible arguments with -f or --remove. * doc/coreutils.texi (cp invocation): Clarify that -f doesn't directly impact the removal of non-traversable symlinks. * tests/cp/dir-rm-dest.sh: Test the new behavior. * tests/cp/thru-dangling.sh: Enforce -f behavior wrt symlinks. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/31335 maint: make chmod/chgrp/chown leak free under valgrind * src/chmod.c: Deallocate the mode change array in dev mode. * src/chown.c: Make chopt_free() actually deallocate, but only call in dev mode. * src/chgrp.c: Likewise. doc: improve formatting of nl --help * src/nl.c (usage): Better delineate the information. 2018-05-14 Paul Eggert who: simplify port to GCC 8 * src/who.c (make_id_equals_comment): Use simpler workaround for GCC bug 85602. Suggested by Martin Sebor in: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85602#c3 2018-05-04 Pádraig Brady build: make GCC 8 adjustments more portable * src/chown-core.h (chopt_free): Just define away this noop. * src/chown-core.c (chopt_free): Remove the empty implementation. 2018-05-04 Paul Eggert build: update gnulib submodule to latest 2018-05-03 Paul Eggert maint: port to GCC 8 * src/chown-core.h (chopt_free, gid_to_name, uid_to_name): No longer const. * src/make-prime-list.c (xalloc): Add malloc attribute. * src/who.c (make_id_equals_comment): Work around GCC bug 85602 by using mempcpy rather than strncat. Although the old code was correct, strncat raises so many hackles that it’s not worth maintaining its use here. maint: remove strpbrk module * bootstrap.conf (gnulib_modules): Remove obsolete module strpbrk. build: update gnulib submodule to latest 2018-04-21 Pádraig Brady doc: retroactively adjust info about tail and closed output * NEWS: Expand on the 8.28 description of how tail more responsively reacts to closed output, and move from "Improvements" to "Changed behavior". * cfg.mk (old_NEWS_hash): Regenerate. Fixes https://bugs.gnu.org/31225 2018-04-19 Pádraig Brady doc: timeout --help: mention 0 DURATION disables timeout * src/timeout.c (usage): Mention that a duration of 0 disables the associated timeout, which is both concise info and useful functionality as timeouts are frequently configured. 2018-04-06 Eric Blake doc: retroactively document -e/-u addition in NEWS Prompted by https://bugs.gnu.org/31045 * NEWS: Update 8.8 blurb to mention other split additions. * cfg.mk (old_NEWS_hash): Regenerate. 2018-04-03 Paul Eggert doc: Clarify octal bits in permissions * doc/perm.texi (Numeric Modes): Briefly explain octal. Reorder description to make it more intuitive (Bug#29069). 2018-03-28 Tobias Stoeckmann cut: improve large file support on 32 bit Increase max range from SIZE_MAX to UINTMAX_MAX, which will allow cut to support line lengths up to the max file size on all systems. The inherent SIZE_MAX limitation in cut was removed with the enhancements in https://bugs.gnu.org/13127. Also numfmt gets similarly increased --field ranges due to shared code. * src/cut.c: s/size_t/uintmax_t/. * src/numfmt.c: Likewise. * src/set-fields.c: Likewise. * src/set-fields.h: Likewise. * tests/misc/cut-huge-range.sh: Adjust accordingly. * tests/misc/numfmt.pl: Likewise. * NEWS: Mention the improvement. 2018-03-28 Pádraig Brady tests: avoid a recent syntax-check failure * tests/ls/a-option.sh: s/framework_failure/&_/. 2018-03-27 Paul Eggert ls: -A now overrides -a Problem reported by Karl Berry (Bug#30963). * NEWS: Mention this. * src/ls.c (decode_switches): Implement this. * tests/ls/a-option.sh: New file. * tests/local.mk (all_tests): Add it. 2018-03-24 Roland Hieber doc: fix two typos in github templates * .github/ISSUE_TEMPLATE.txt: Fix typo "coreitils" in the URL to the bug tracker. * .github/PULL_REQUEST_TEMPLATE.txt: Likewise. 2018-03-16 Pádraig Brady ls: increase the allowed abmon width from 5 to 12 This will impact relatively few languages, and will make Arabic or Catalan etc. output unambiguous abbreviated month names. * src/ls.c (MAX_MON_WIDTH): Increase from 5 to 12. * NEWS: Mention the bug fix. * tests/ls/abmon-align.sh: Augment to check for ambiguous output. Fixes https://bugs.gnu.org/30814 2018-03-14 Brent Petit stat,tail: add support for the EXFS file system Enhanced XFS (EXFS) is a version of XFS maintained by HPE. EXFS uses a unique magic number to allow the use of community XFS, and EXFS filesystems at the same time. * src/stat.c (human_fstype): Add file system ID definition, and use "exfs" as the name. * NEWS: Mention the Improvement. 2018-03-06 Paul Eggert build: update gnulib submodule to latest 2018-03-05 Paul Eggert stat: work around IBM xlC bug Problem reported by John Wiersba (Bug#30718) * src/stat.c (human_time): Avoid giving an integer constant expression a name, as it runs afoul of a bug in IBM XL C/C++ for AIX 12.01.0000.0002. 2018-03-04 Bernhard Voelker maint: adjust email address of Keith Thompson in THANKS.in * THANKS.in (Keith Thompson): Update email address as requested by himself at https://lists.gnu.org/r/coreutils/2018-03/msg00004.html 2018-02-25 Pádraig Brady cp: set appropriate default permissions for special files This issue was introduced in commit v8.19-145-g24ebca6 * src/copy.c (copy_internal): When setting default permissions to use with --no-preserve=mode, only set executable bits for directories or sockets. * NEWS: Mention the fix. * tests/cp/preserve-mode.sh: Add a test case. Fixes https://bugs.gnu.org/30534 2018-01-21 Pádraig Brady doc: use consistent example format in manual * doc/coreutils.texi: Use @example consistently as we don't need the smaller or fixed width representation. This is especially true for the synopsis of commands. @smallexample is rendered left aligned for HTML which is awkward to read with the center aligned main content. 2018-01-10 Paul Eggert mv: clarify ‘mv -n A A’ change mv: fewer syscalls for ‘mv a b’ This builds on a previous patch for mv atomicity (Bug#29961). It merely improves performance; it does not fix bugs. * src/copy.h (struct cp_options): New members last_file, rename_errno. * src/copy.c (copy_internal): Support new rename_errno member for the copy options. Avoid calling stat when new members suggest it’s not needed. (cp_options_default): Initialize new members. * src/mv.c: Include renameat2.h. (main): With two arguments, first call ‘renamat2 (AT_FDCWD, "a", AT_FDCWD, "b", RENAME_NOREPLACE)’. Use its results to skip remaining processing if possible; for example, if it succeeds there is no need to stat either "a" or "b". Also, set x.last_file when it is the last file to rename. mv: improve -n atomicity Problem reported by Kamil Dudka (Bug#29961). * NEWS: Mention this. * src/copy.c: Include renameat2.h. (copy_internal): If mv, try renameat2 first thing, with RENAME_NOREPLACE. If this works, skip most of the remaining code. Also, fail quickly if it fails with EEXIST, and we are using -n. 2018-01-10 Michael Orlitzky doc: warn about following symlinks recursively in chown/chgrp In both chown and chgrp (which shares its code with chown), operating on symlinks recursively has a window of vulnerability where the destination user or group can change the target of the operation. Warn about combining the --dereference, --recursive, and -L flags. * doc/coreutils.texi (warnOptDerefWithRec): Add macro. (node chown invocation): Add it to --dereference and -L. (node chgrp invocation): Likewise. See also: CVE-2017-18018 2018-01-06 Paul Eggert cp: remove ASSIGN_BASENAME_STRDUPA * src/cp.c (do_copy): Just use ASSIGN_STRDUPA, as this simplifies the code and uses less memory. 2018-01-04 Paul Eggert mv: -n overrides -u * NEWS: Mention these fixes. * doc/coreutils.texi (cp invocation, mv invocation): Mention that -n is silent, and that it overrides -u. * src/cp.c, src/mv.c (main): -n overrides -u. 2018-01-03 Paul Eggert tr: add -A, for compatibility with AIX tr Problem reported by Michael (Bug#29946). * src/tr.c (main): Add undocumented -A option. 2018-01-03 Michael Orlitzky doc: clarify chown/chgrp --dereference defaults * doc/coreutils.texi: the documentation for the --dereference flag of chown/chgrp states that it is the default mode of operation. Document that this is only the case when operating non-recursively. 2018-01-02 Pádraig Brady tests: avoid false failure with xargs on AIX * tests/misc/shred-remove.sh: AIX xargs defaults to using '_' to indicate end of input, thus ignoring it. Rather than specifying -E to avoid this behavior, simplify by removing sed and xargs usage. 2018-01-01 Pádraig Brady maint: update all copyright year number ranges Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year. 2017-12-27 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.29 * NEWS: Record release date. 2017-12-23 Pádraig Brady tests: avoid false failure on AIX 7.2 * tests/tail-2/pipe-f.sh: Close stdout in a subshell to ensure the current shell isn't impacted. Subsequent piped commands like `echo foo | blah` were seen to fail due to the previous closing of stdout. Reported by Assaf Gordon. doc: describe recent build checks for 32 bit time_t * README: Document the new handling of 32 bit time_t, with examples of how to build in 64 bit mode on AIX. Also mention that GNU make is desired on AIX due to its mishandling of the "[" target. Suggested by Assaf Gordon. 2017-12-21 Pádraig Brady tests: fix recent portability issues on solaris 10 * tests/misc/ptx.pl: Escape the '^' character which is otherwise considered as a line continuation character. * tests/misc/shred-remove.sh: sed doesn't support \n. maint: remove reference to excluded changelog item * build-aux/git-log-fix: Remove old entry. 2017-12-20 Pádraig Brady maint: add doc/coverage to .gitignore * .gitignore: Ignore the generated coverage report. doc: remove older ChangeLog items * Makefile.am: Update the oldest documented version to 8.20 which is now about 5 years old. 2017-12-18 Bernhard Voelker doc: mention which privileges are needed to chmod POSIX specification for chmod(1): https://pubs.opengroup.org/onlinepubs/9699919799/utilities/chmod.html * doc/coreutils.texi (chmod invocation): Add a sentence about who can change the file mode bits of a file - (almost) a copy from what POSIX requires. Fixes https://bugs.gnu.org/29207. 2017-12-16 Pádraig Brady tests: fix recent regressions with dash * tests/misc/timeout.sh: dash outputs the "Killed" message to stderr rather than the terminal. * tests/misc/usage_vs_getopt.sh: dash doesn't yet support the POSIX proposed $'...' shell quoting syntax. build: avoid a signed overflow warning in ptx * src/ptx.c (fix_output_parameters): GCC 6.3.1 with ./configure --enable-single-binary would give: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow] if (file_index > 0) So change the type of file_index to signed (size_t). 2017-12-11 Bernhard Voelker maint: adjust for the renamed nstrfime gnulib module * bootstrap.conf: s/strftime/nstrfrime/. 2017-12-11 Pádraig Brady build: update gnulib submodule to latest * gnulib: Update with various build/test fixes. tests: fix false failure in new dd/nocache_eof test * test/dd/nocache_eof.sh: Also handle fadvise64_64 which is used on 32 bit x86. Note strace internally maps fadvise64_64 to {arm,xtensa}_fadvise64_64. tail: fix tailing non seekable files on certain systems * src/tail.c (tail_bytes): On systems were blksize_t is unsigned and the same size or wider than off_t (android for example), our initialized (off_t) -1 would be promoted to unsigned before comparison, and thus fail to follow the appropriate path. * tests/tail-2/tail-c.sh: Add a test case. * NEWS: Mention the fix. This issue was introduced in commit v8.23-47-g2662702 Reported at https://github.com/termux/termux-app/issues/233 build: avoid build failure without sys/mtio.h * m4/jm-macros.m4: Check for the header. * src/dd.c: Avoid the workaround where the header is not available (on non glibc systems). * src/shred.c: Likewise. doc: reorganize ls -k and --time-style help * src/ls.c (usage): Clarify -k only applies to -s usage and directory 'total' lines. Move the description of TIME_STYLE out of the option section as it was awkward to read and write there within 80 columns. 2017-12-10 Pádraig Brady doc: clarify numeric setuid handling in chmod man page * man/chmod.x: Update the information to state one can clear the setuid and setgid bits for directories numerically using an additional leading '0' or a leading '='. That has been supported since v8.15-64-g8931cdb. Fixes https://bugs.gnu.org/29390 doc: shred: change 'truncate' to the more descriptive 'deallocate' * doc/coreutils.texi (shred invocation): s/truncate/deallocate/. * src/shred.c (usage): Likewise. Fixes https://bugs.gnu.org/29317 doc: clarify that cp --force may recreate files * doc/coreutils.texi (cp invocation): The language used to describe recreating the file was a little confusing as it mentioned opening a removed file. Fixes https://bugs.gnu.org/29315 2017-12-04 Kamil Dudka doc: fix default QUOTING_STYLE for %N format of stat(1) * doc/coreutils.texi (stat invocation): The default value of QUOTING_STYLE for the %N format of 'stat --printf' is 'shell-escape-always'. Fixes https://bugs.gnu.org/29563 Reported by Christian Groessler at https://bugzilla.redhat.com/1520399#c3 2017-12-02 Jean Delvare tests: make ls/block-size more readable * tests/ls/block-size.sh: The output of the test was hard to read. Add comments saying what we are testing to make it easier to understand. 2017-11-29 Bernhard Voelker Pádraig Brady tests: verify usage vs. getopt Verify that all options mentioned in usage are actually recognized by the program. * tests/misc/usage_vs_getopt.sh: Add test. * tests/local.mk (all_tests): Reference it. 2017-11-29 Pádraig Brady readlink: remove superfluous comma from usage output * src/readlink.c (usage): Remove ',' after --quiet option. 2017-11-29 Bernhard Voelker all: use consistent diagnostics for unknown long options Previously, e.g. cksum failed to output the offending unknown long option: $ cksum --unknown-opt cksum: invalid option -- '-' Try 'cksum --help' for more information. i.e., it tried to diagnose '-' as short option. Instead, it should diagnose the unknown long option: $ cksum --unknown-opt cksum: unrecognized option '--unknown-opt' Try 'cksum --help' for more information. * src/cksum.c (long_options): Add struct with null entry only. (main): Use it in the getopt_long call. * src/dd.c: Likewise. * src/hostid.c: Likewise. * src/hostname.c: Likewise. * src/link.c: Likewise. * src/logname.c: Likewise. * src/nohup.c: Likewise. * src/sleep.c: Likewise. * src/tsort.c: Likewise. * src/unlink.c: Likewise. * src/uptime.c: Likewise. * src/users.c: Likewise. * src/whoami.c: Likewise. * src/yes.c: Likewise. * NEWS (Improvements): Mention the fix. 2017-11-29 Pádraig Brady test: fix issues with tests/cp/preserve-mode.sh * tests/cp/preserve-mode.sh: This was the only use of awk, which may not be available on the system resulting in an ineffective test. Also the permissions bits for directories were not being checked at all. build: update gnulib submodule to latest * gnulib: Update with various build/test fixes. 2017-11-28 Pádraig Brady build: update gnulib submodule to latest * gnulib: Update including various build fixes. 2017-11-27 Bernhard Voelker timeout: also support short -v option * src/timeout.c (main): Add short option character 'v' to getopt_long call. * tests/misc/timeout.sh: Run the test both for the long and the short option. 2017-11-25 Pádraig Brady dd: support iflag=direct with arbitrary sized files * src/dd.c (iread): Handle read error with a non-aligned file offset in the O_DIRECT case. This is not an issue on XFS at least, but on EXT4 the final read will return EINVAL rather than the expected 0 to indicate EOF. * tests/dd/direct.sh: Test the iflag=direct case also. * NEWS: Mention the improvement. 2017-11-24 Pádraig Brady timeout: add --verbose to diagnose timeouts This is useful as handling in shell is complicated with the varying exit status in the --kill-after case. * src/timeout.c (main): Handle '-v' and store COMMAND for the diagnostic. (cleanup): Diagnose the signal name before sending. (usage): Document -v, --verbose. * doc/coreutils.texi (timeout invocation): Likewise. * tests/misc/timeout.sh: Add a test case. * NEWS: Mention the new feature Fixes https://bugs.gnu.org/21760 2017-11-19 Pádraig Brady tail: seek to the end of block devices * src/tail.c (tail_bytes): Try lseek(..., SEEK_END) when we can't determine the file size. * tests/tail-2/end-of-device.sh: Add a new root only test. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. Paul Eggert suggested using lseek() (rather than ioctl(BLKGETSIZE64)). Fixes https://bugs.gnu.org/29259 2017-11-14 Bernhard Voelker maint: include the module year2038 from gnulib * bootstrap.conf (gnulib_modules): Add 'year2038' to ensure that time_t is 64-bit (and thus works after 2038). Suggested by Bruno Haible in https://lists.gnu.org/r/bug-gnulib/2017-11/msg00022.html 2017-11-14 Bernhard Voelker maint: update gnulib to latest * gnulib: Update - mainly for the recent year2038 changes. * tests/init.sh: Update from gnulib/tests/init.sh. 2017-11-09 Assaf Gordon doc: add github issue/pull-request templates These templates instruct contributors not to use github, and instead use the upstream GNU development resources. Discussed in http://lists.gnu.org/archive/html/coreutils/2017-11/msg00007.html . * .github/ISSUE_TEMPLATE.txt, .github/PULL_REQUEST_TEMPLATE.txt: New files. 2017-11-08 Jim Meyering maint: make hook script reject "/archive/html" in lists.gnu.org URLS * scripts/git-hooks/commit-msg: Require the abbreviated "/r/" form in any log message URL. maint: shorten https://lists.gnu.org/archive/html/... links Each /archive/html/ part can be replace with /r/. Run this to induce the change: git grep -l archive/html|xargs perl -pi -e 's,/archive/html/,/r/,g' * TODO: Perform that substitution. * bootstrap: Likewise. * src/sort.c (sequential_sort): Likewise. * src/tail.c (tail_file): Likewise. * tests/misc/sort-merge-fdlimit.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise. * tests/misc/unexpand.pl: Likewise. * tests/rm/readdir-bug.sh: Likewise. * tests/tail-2/inotify-rotate.sh: Likewise. 2017-11-07 Thomas Deutschmann tests: avoid false failure with inaccessible mount points * tests/ls/readdir-mountpoint-inode.sh: Skip the test if any mount points are inaccessible by the current user. Fixes https://bugs.gnu.org/29167 Reported at: https://bugs.gentoo.org/353164 2017-11-06 Bernhard Voelker doc: fix "Up" field of realpath usage examples Older versions of 'makeinfo' choke on a missing reference: ./doc/coreutils.texi:14177: `Realpath usage examples' has no Up field\ (perhaps incorrect sectioning?). makeinfo: Removing output file `doc/coreutils.info' due to errors; \ use --force to preserve. * doc/coreutils.texi (realpath invocation): Add a menu referencing the usage examples - introduced in v8.27-91-g7449f0d. 2017-11-06 Pádraig Brady maint: ensure https:// URLs are used in --help and man pages * configure.ac(AC_INIT): Specify the URL explicitly, so we're not dependent on unreleased autoconf. 2017-10-31 Assaf Gordon stat: output default formats for --terse in usage Suggested by L A Walsh in https://bugs.gnu.org/28763 . * src/stat.c (fmt_terse_fs): Define format for --terse -f here. (fmt_terse_regular): Define format for --terse here. (fmt_terse_selinux): Likewise for when SELinux is enabled. (default_format): Use the above constants. (usage): Output the formats for the terse modes. 2017-10-30 Pádraig Brady df: fix hang with fifo argument * src/df.c (main): stat() before open(), and avoid the optional open when given a fifo argument. * tests/df/unreadable.sh: Add a test case. * NEWS: Mention the fix. Fixes https://bugs.gnu.org/29038 2017-10-28 Jim Meyering build: ls.c: apply _GL_ATTRIBUTE_PURE to more functions * src/ls.c (DEFINE_SORT_FUNCTIONS): Apply _GL_ATTRIBUTE_PURE to each strcmp-derived function definition, since GCC8 with -Wsuggest-attribute=pure now warns it is needed. 2017-10-26 Vincent Lefevre doc: reference statfs(2) in the stat(1) man page * man/stat.x (SEE ALSO): Mention statfs(2) in addition to stat(2). Note statfs() is generally used rather than statvfs(), so we'll defer that reference to the SEE ALSO section of statfs(2). Fixes https://bugs.gnu.org/28989 2017-10-25 Pádraig Brady tests: avoid false failure when O_DIRECT isn't supported * tests/dd/nocache_eof.sh: Only run the O_DIRECT tests when 512 byte alignment is supported. Otherwise with older XFS on systems with > 1MiB pages, or on file systems not supporting O_DIRECT, there would have been false failures. * tests/dd/direct.sh: Clarify the skip message. 2017-10-25 Pádraig Brady dd: fix nocache regions passed to posix_fadvise() Previously with oflag=direct the call to invalidate_cache() was not passed to the kernel, as it was less than a page size, and a subsequent call was not made to invalidate the pending space. Similarly with oflag=nocache the pending space at EOF was not invalidated. Even though these amount to only a single page in the page cache it can be significant. For example on XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files would have been read inefficiently if any pages were cached, even if they were already synced to storage. * src/dd.c (i_nocache_eof, o_nocache_eof): New bools used to control when we want invalidate_cache(,0) to clear to EOF. (cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize calls to the relatively expensive advise function, rather than page_size. This also makes it clear that while the kernel function operates on pages, this size is chosen for performance reasons. (invalidate_cache): Refactor to share more code between input and output paths. Use i_nocache_eof and o_nocache_eof rather than proxying off max_records. Ensure we invalidate full pages when clearing to EOF as the kernel will ignore any non complete pages. Fix the offset used for the output path. (dd_copy): Invalidate the cache of the input after the offset is updated, for consistency and so we don't try to invalidate before the start of the file. When we read EOF on input, set flags so that we invalidate to EOF. (main): Invalidate to EOF in more cases, by depending on the i_nocache_eof and o_nocache_eof flags. * doc/coreutils.texi (dd invocation): Clarify the alignment and persisted caveats on the example applying "nocache" to part of a file. * tests/dd/nocache_eof.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. Issue reported by Eric Bergen. 2017-10-24 Michael Stone doc: mention QUOTING_STYLE env var in ls man page * src/ls.c (usage): Mention QUOTING_STYLE with the --quoting-style option, and indicate it has lower precedence than that option. 2017-10-24 Pádraig Brady maint: apply suggested cleanup to recent stty.c change This should have been part of commit v8.28-17-gf926f7c * src/stty.c (check_argument): Align line continuation chars, and ensure the function macro is immune to usage with if/else. Suggested by Jim Meyering and Paul Eggert. b2sum: fix crash with --check and truncated input * src/md5sum.c (split_3): Ensure we don't walk off the end of the string. * tests/misc/b2sum.sh: Add test cases. Fixes https://bugs.gnu.org/28860 2017-10-24 Pádraig Brady stty: fix processing of options when -F is specified This was a latent issue that became significant with the addition of the -F option in FILEUTILS-3_16n-56-ge46a424 * src/stty.c (apply_settings): Refactor argument checking to a function macro. Augment the argument check to ignore NULLed out arguments (already processed -F). * NEWS: Mention the fix. * tests/misc/stty-invalid.sh: Add a test case. Fixes https://bugs.gnu.org/28859 2017-10-24 Pádraig Brady timeout: fix a small race that would ignore command exit This fixes a regression from commit v8.26-39-g2f69dba * src/timeout.c (block_cleanup_and_chld): Rename from block_cleanup to indicate we also block SIGCHLD to avoid the race where SIGCHLD fires between waitpid() polling and sigsuspend() waiting for a signal. * NEWS: Mention the fix. 2017-10-24 Thomas Jarosch timeout: fix regression when invoked with blocked SIGCHLD We inherit the signal mask from our parent process, therefore ensure SIGCHLD is not blocked. If SIGCHLD is blocked, sigsuspend() won't be interrupted when the child process exits and we hang until the timeout (SIGALRM). This fixes a regression from commit v8.26-39-g2f69dba * src/timeout.c (install_sigchld): Ensure SIGCHLD is unblocked. * NEWS: Mention the issue. 2017-10-02 Pádraig Brady build: reinstate distribution of man pages man pages change little between systems, so falling back to distributed pages make sense when cross compiling or lacking perl. * man/local.mk: Add all man pages to EXTRA_DIST so that they're distributed in the generated tarball. Use the dummy-man page generator if cross compiling. Set TZ to avoid a distcheck failure where man pages used a diffent month than those rebuilt (with a .timestamp). * man/dummy-man: Only fall back to generating a stub if copying an existing man page fails. * man/help2man: Sync portable TZ=UTC0 specification from upstream help2man. * NEWS: Mention the build-related change. Fixes https://bugs.gnu.org/28574 2017-10-02 Pádraig Brady maint: remove a duplicate entry from THANKS * .mailmap: Prefer Colin Watson's last used email address. 2017-09-25 Paul Eggert copy: revert recent patch for vulnerable dirs I plan to propose a better patch to catch vulnerable parent directories. * NEWS, doc/coreutils.texi (Target directory): Document this. * src/cp.c, src/install.c, src/ln.c, src/mv.c: Do not include targetdir.h. (target_directory_operand): Remove test for vulnerable parents. * src/cp.c (stat_target_operand): Remove. All uses removed. * src/local.mk (noinst_HEADERS): Remove src/targetdir.h. (src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES) (src_mv_SOURCES): Remove src/targetdir.c. * src/targetdir.c, src/targetdir.h: Remove. * tests/mv/vulnerable-target.sh: Remove. * tests/local.mk (all_root_tests): Remove it. 2017-09-24 Pádraig Brady tests: fix test hang on case insenitive file systems * tests/split/filter.sh: Due to an invalid 'FILE = zero.in' construct trying to initialize a FILE variable, it would instead try to run the FILE command which is present on macOS 10.13 with APFS. We also remove a redundant duplicate test clause introduced during a rebase, and simplify the piped timeout command, to avoid requiring a subshell and associated quoting. * THANKS.in: Add the reporter Jack Howarth. Fixes https://bugs.gnu.org/28506 2017-09-21 Pádraig Brady tests: avoid a false failure in expr test with UTF8 * tests/misc/expr.pl: Skip the quote varying tests in the multi-byte locales as these tests aren't that interesting in those locales. Also ERR_SUBST is already defined for some tests so awkward to redefine to munge UTF8 quotes to ASCII. 2017-09-20 Assaf Gordon expr: add detailed syntax error messages Show offending argument instead of a generic 'syntax error' message. Suggested by Bernhard Voelker in https://bugs.gnu.org/28461#13 . * src/expr.c (syntax_error): Remove. (required_more_args): New function. (eval7, main): Replace syntax_error call with detailed die message. * tests/misc/expr.pl: Add tests for new messages. 2017-09-20 Pádraig Brady maint: fix new syntax-check failures from HTTPS adjustments * cfg.mk [old_NEWS_hash]: update with `make update-NEWS-hash`. [sc_long_lines]: Avoid flagging (long) URLs in NEWS. * src/sort.c: Tweak to a shorter line. * src/tail.c: Likewise. Introduced in v8.28-4-gbe87d61 maint: fix new syntax check failures from copy restrictions * doc/coreutils.texi: Remove doubled word. * src/targetdir.c: Explicitly mark exported function. * tests/local.mk: This is not a root only test. * tests/mv/vulnerable-target.sh: Use returns_. Introduced in v8.28-3-g44ccd1c shred: reinstate --remove file name length obfuscation This was unintentionally removed in v8.27-60-g2ae1460 * src/shred.c (wipename): Interate through all name lengths. * tests/misc/shred-remove.sh: Add test cases. * NEWS: Mention the bug fix. Fixes https://bugs.gnu.org/28507 2017-09-19 Paul Eggert maint: copy bootstrap from Gnulib all: prefer HTTPS in URLs copy: check for vulnerable target dirs * NEWS, doc/coreutils.texi (Target directory): Document this. * src/cp.c, src/install.c, src/ln.c, src/mv.c: Include targetdir.h. (target_directory_operand): Use the new targetdir_operand_type function to check for vulnerable target directories. * src/cp.c (stat_target_operand): New function. (target_directory_operand, do_copy): Use it. * src/local.mk (noinst_HEADERS): Add src/targetdir.h. (src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES) (src_mv_SOURCES): Add src/targetdir.c. * src/targetdir.c, src/targetdir.h: New files. * tests/mv/vulnerable-target.sh: New test. * tests/local.mk (all_root_tests): Add it. 2017-09-14 Bernhard Voelker ptx: avoid infloop due to zero-length matches with -S regex * src/ptx.c (find_occurs_in_text): Die with an appropriate error diagnostic when the given regular expression returns a match of length 0. * tests/misc/ptx.pl (S-infloop): Add a test. * NEWS (Bug fixes): Mention the fix. Fixes https://bugs.gnu.org/28417 which was detected using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University. 2017-09-02 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.28 * NEWS: Record release date. 2017-09-01 Pádraig Brady tests: fix false failure in recent ls --hyperlink test * tests/ls/hyperlink.sh: If the hostname or any part of the absolute path would be changed due to URL encoding, the test would fail. Therefore simplify to remove these components of the URL from consideration. maint: avoid a syntax-check failure * .gitignore: Remove lines indicated by sc_gitignore_redundant in a freshly checked out repo. 2017-08-31 Pádraig Brady tests: exclude the expensive gnulib fts-tests * gnulib: The only change in this gnulib update is the tagging of the fts-tests module as longrunning, which gnulib-tool currently implicitly excludes. This test was seen to take about 20s and 285MB. Reported by Assaf Gordon on space restricted VMs. tty: don't distinguish input errors * src/tty.c (main): Don't distinguish ENOTTY from other errors, because isatty() doesn't portably distinguish errors. Solaris returns ENOENT for all input errors for example. Musl also returns ENOENT, and ENODEV may be returned as disscussed at: http://openwall.com/lists/musl/2017/04/06/6 * tests/misc/tty.sh: Adjust accordingly. tests: avoid printf '0*d' construct unsupported by ash * tests/ln/sf-1.sh: Generate specific length with space padding which is supported. Reported by Assaf Gordon on Alpine Linux. 2017-08-31 Pádraig Brady tests: skip tests upon failure to set SELinux context On some setups the root:object_r:tmp_t context is invalid. This does indicate a limitation in the test framework, but for now we'll relax this to skipping the tests. The tests still run on a Fedora 25 system for example. * tests/cp/cp-a-selinux.sh: Upon chcon error, skip rather than ERROR. * tests/install/install-Z-selinux.sh: Likewise. * tests/misc/chcon.sh: Likewise. * tests/misc/runcon-no-reorder.sh: Likewise. * tests/misc/selinux.sh: Likewise. * tests/mkdir/restorecon.sh: Likewise. 2017-08-30 Kamil Dudka expr: fix a recently introduced memory leak * src/expr.c (eval6): Free memory allocated by mbs_logical_substr(). Introduced in v8.27-47-ga9f2be5. Detected by Coverity Analysis: Error: RESOURCE_LEAK: src/expr.c:851: leaked_storage: Variable "s" going out of scope leaks the storage it points to. 849| char *s = mbs_logical_substr (l->u.s, pos, len); 850| v = str_value (s); 851|-> } 852| freev (l); 853| freev (i1); 2017-08-30 Pádraig Brady build: fix build of renameat2 on Alpine Linux * gnulib: The only change included in this update it the added check for the presence of which is not present on Alpine Linux by default. tty: fix exit code with EINVAL * src/tty.c (main): All systems mention that isatty() man return EINVAL as well as (the POSIX compliant) ENOTTY. Also Centos 6 was seen to return EINVAL from ttyname(). * tests/misc/tty.sh: Fix a test issue where we assume standard input is always a valid tty. Reported by Assaf Gordon on OpenSolaris 5.10 and 5.11, and Centos 6.5 2017-08-30 Pádraig Brady runcon: revert "disable use of the TIOCSTI ioctl" This reverts commit v8.27-97-g8cb06d4 because the setsid() fallback was not implemented correctly and disabling the ioctl was not a complete solution to the security issue of the child being passed the tty of the parent. Given runcon is not really a sandbox command, the advice is to use `runcon ... setsid ...` to avoid this particular issue. 2017-08-30 Pádraig Brady stat: fix determination of max name length on BSD systems We only use one of statfs or statvfs for `stat -f` and on the BSDs we use statfs which doesn't have the f_namelen member. However on OpenBSD and later FreeBSD systems statfs does provide f_namemax, so use that. * NEWS: Mention the improvement for OpenBSD and FreeBSD. * m4/stat-prog.m4: Check for f_namemax in the statfs struct. * src/stat.c: Return '?' rather than '*' when we can't determine the max length of the file system. * tests/ln/sf-1.sh: This test was failing on all BSDs due to '*' being returned for the max length which caused the test to attempt to create 1Mi+1 names. The test now uses a short name when we can't determine the max name length to use. Reported by Assaf Gordon on various BSD based systems. 2017-08-29 Pádraig Brady stat,tail: support "AAFS" AppArmor file system * src/stat.c (human_fstype): This file system is used to manage AppArmor policy in the Linux kernel. all: update gnulib submodule to latest * bootstrap: Sync timestamp update. 2017-08-29 Pádraig Brady runcon: disable use of the TIOCSTI ioctl Similar to the issue with SELinux sandbox (CVE-2016-7545), children of runcon can inject arbitrary input to the terminal that would be run at the originating terminal privileges. The new libseccomp dependency is widely available and used on modern SELinux systems, but is not available by default on older systems like RHEL6 etc. * m4/jm-macros.m4: Check for libseccomp and warn if unavailable on selinux supporting systems. * src/local.mk: Link runcon with -lseccomp. * src/runcon.c (disable_tty_inject): A new function to disable use of the TIOCSTI using libseccomp, or with setsid() where libseccomp is unavailable. * tests/misc/runcon-no-inject.sh: A new test that uses python to make the TIOCSTI call, and ensure that doesn't succeed. * tests/local.mk: Reference the new test * NEWS: Mention the fix. Addresses http://bugs.gnu.org/24541 2017-08-29 Pádraig Brady ls: support --hyperlink to output file:// URIs Terminals such as iTerm2 and VTE based terminals (as of version 0.49.1), support hyperlinks when passed terminals codes as described at: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda * src/ls.c (gobble_file): Allocate an absolute file name to output. (quote_name): Output the absolute name with the appropriate codes. (file_escape): A new function to encode file names as per rfc8089. (main): Handle the new option and call the file_escape_init() helper. Disable --dired when --hyperlink is specified. (print_dir): Get the absolute file name here too, so that the directory name can be linkified. * NEWS: Mention the new feature. * tests/ls/hyperlink.sh: Add a new test. * tests/local.mk: Reference the new test. * doc/coreutils.texi (ls invocation): Describe --hyperlink. 2017-08-29 Pádraig Brady doc: remove older ChangeLog items This saves about 0.5MB uncompressed from the tarball. * Makefile.am: Following on from v8.26-34-g2c64bc8 update the oldest documented version to 8.18 which is now about 5 years old. Also remove older ChangeLogs that were previously thought to be for changes not in the git history, but are adequately recorded upon review. * build-aux/ChangeLog-2007: Remove file. * lib/ChangeLog-2007: Likewise. * m4/ChangeLog-2007: Likewise. 2017-08-29 Colin Watson env: add --chdir option This is useful when chaining with other commands that run commands in a different context, while avoiding using the shell to cd, and thus having to consider shell quoting the chained command. * NEWS (New features): Document the new option. * doc/coreutils.texi (env invocation): Likewise. * src/env.c (usage): Likewise. (main): Implement the new option. * tests/misc/env.sh: Test the new option. 2017-08-29 Pádraig Brady tests: don't fail tests when failing to write files * tests/sample-test: Use framework_error_ rather than fail=1 * tests/chown/deref.sh: Likewise. * tests/chown/preserve-root.sh: Likewise. * tests/cp/src-base-dot.sh: Likewise. * tests/dd/unblock-sync.sh: Likewise. * tests/du/2g.sh: Likewise. * tests/du/inacc-dest.sh: Likewise. * tests/du/one-file-system.sh: Likewise. * tests/fmt/goal-option.sh: Likewise. * tests/ln/hard-backup.sh: Likewise. * tests/ls/color-dtype-dir.sh: Likewise. * tests/ls/m-option.sh: Likewise. * tests/ls/stat-dtype.sh: Likewise. * tests/ls/time-style-diag.sh: Likewise. * tests/ls/x-option.sh: Likewise. * tests/misc/chcon.sh: Likewise. * tests/misc/nohup.sh: Likewise. * tests/misc/od-N.sh: Likewise. * tests/misc/sort-compress.sh: Likewise. * tests/misc/tac-continue.sh: Likewise. * tests/misc/time-style.sh: Likewise. * tests/mv/backup-dir.sh: Likewise. * tests/mv/dir2dir.sh: Likewise. * tests/rm/dir-no-w.sh: Likewise. * tests/rm/dir-nonrecur.sh: Likewise. * tests/rm/inaccessible.sh: Likewise. * tests/rm/interactive-always.sh: Likewise. * tests/rm/interactive-once.sh: Likewise. * tests/rm/rm3.sh: Likewise. * tests/rm/v-slash.sh: Likewise. * tests/touch/relative.sh: Likewise. 2017-08-29 Josef Cejka Bernhard Voelker df: avoid stat() for dummy file systems with -l When systemd is configured to automount a remote file system - see 'man systemd.automount(5)', then the mount point is initially mounted by systemd with the file system type "autofs". When the resource is used later on, then the wanted file system is mounted over that mount point on demand. 'df -l' triggered systemd to mount the file system because it called stat() on the mount point. Instead of single-casing "autofs" targets, we can avoid stat()ing all dummy file systems (which includes "autofs"), because those are skipped later on in get_dev() anyway. *src/df.c (filter_mount_list): Also skip dummy file systems unless the -a option or a specific target are given. * NEWS: Mention the fix. Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059 2017-08-29 Assaf Gordon doc: add 'realpath usage examples' section * doc/coreutils.texi (Realpath usage examples): New section. 2017-08-29 Assaf Gordon doc: fix realpath index entry The 'readlink' node has '@findex realpath' in it. This results in info doc/coreutils.info realpath incorrectly jumping to the 'readlink' node (instead of the 'realpath' node). Change it to @cindex instead. * doc/coreutils.texi (readlink): Change '@findex realpath' to @cindex. 2017-08-29 Assaf Gordon realpath: improve usage description for --relative-{to,base} * src/realpath.c (usage): Explicitly say 'DIR' instead of 'FILE' for --relative-{to,base} parameters, to avoid giving the impression that regular files can be used as relative base. * doc/coreutils.texi (realpath): Same. 2017-08-25 Pádraig Brady ls: consistently quote symlink targets * src/ls.c (gobble_file): Disable the optimization to avoid quoting if the symlink target itself needs quoting. This was introduced with the quoting alignment adjustments in v8.25-106-g01971c0 * tests/ls/symlink-quote.sh: Add a test. * tests/local.mk: Reference the test. * NEWS: Mention the fix. 2017-08-25 Pádraig Brady tail: reinstate inotify use with FIFOs commit v8.27-44-g18f6b22 was too aggressive in only allowing inotify use with regular files. This will support responsive processing of `tail -f fifo | ...` * src/tail.c (any_non_regular): Adjust to allow FIFOs since inotify supports these well. * tests/tail-2/inotify-only-regular.sh: Adjust comment. 2017-08-19 Pádraig Brady maint: avoid a syntax check failure * src/sort.c: Don't include stdio--.h as fopen() is no longer used. tests: fix issues on alpine linux * tests/misc/seq-epipe.sh: Remove stale comment. * tests/misc/sort-debug-warn.sh: musl doesn't indicate a set_locale() failure with missing locales, so avoid a test portion in that case. * tests/misc/wc-files0.sh: Avoid a bug on older ash implementations. Addresses http://bugs.gnu.org/28054 2017-08-17 Paul Eggert ptx: fix some integer overflow bugs Problem reported by Lukas Zachar at: http://bugzilla.redhat.com/1482445 * src/ptx.c (line_width, gap_size, maximum_word_length) (reference_max_width, half_line_width, before_max_width) (keyafter_max_width, truncation_string_length, compare_words) (compare_occurs, search_table, find_occurs_in_text, print_spaces) (fix_output_parameters, define_all_fields): Use ptrdiff_t, not int, for object offsets and sizes. (WORD, OCCURS): Use ptrdiff_t, not short int. (WORD_TABLE, number_of_occurs, generate_all_output): Prefer ptrdiff_t to size_t where either will do. (total_line_count, file_line_count, OCCURS, fix_output_parameters) (define_all_fields): Use intmax_t, not int, for line counts. (DELTA): Remove. All uses changed. (OCCURS, find_occurs_in_text, fix_output_parameters): Use int, not size_t, for file indexes. (tail_truncation, before_truncation, keyafter_truncation) (head_truncation, search_table, define_all_fields) (generate_all_output): Use bool for booleans. (digest_word_file, find_occurs_in_text): Use x2nrealloc instead of checking for overflow by hand. (find_occurs_in_text, fix_output_parameters, define_all_fields): Omit unnecessary cast. (fix_output_parameters): Don’t assume integers fit in 11 digits. (fix_output_parameters, define_all_fields): Use sprintf return value rather than calling strlen. (define_all_fields): Do not rely on sprintf to generate a string that may contain more than INT_MAX bytes. (main): Use xstrtoimax, not xstrtoul. Use xnmalloc to catch integer overflow. nohup: simplify by using fcntl * src/nohup.c: Do not include cloexec.h. (main): Use fcntl rather than dup + set_cloexec_flag. sort: use pthread_sigmask, not sigprocmask POSIX says sigprocmask has unspecified behavior in a multithreaded program like ‘sort’. * src/sort.c (pthread_sigmask) [GNULIB_defined_pthread_functions]: New macro, for use when ‘sort’ is not multithreaded. (cs_enter, cs_leave): Use it. Pass address, not value, as this is typically a tad faster. All callers changed. sort: minor cleanups * src/sort.c (move_fd): Rename from move_fd_or_die, since it no longer can die. sort: file descriptor discipline Use O_CLOEXEC when creating file descriptors, so that subsidiary processes do not inherit file descriptors that they do not need. This is helpful for ‘sort’, as it is a multithreaded program that forks and execs. * bootstrap.conf (gnulib_modules): Add mkostemp, open, pipe2. * src/sort.c (create_temp_file): Open temporary file with O_CLOEXEC. (stream_open): Open the stream with O_CLOEXEC. (pipe_fork): Create the pipe with O_CLOEXEC. (check_output): Open the output file with O_CLOEXEC. (main): Use xfopen/xfclose to handle --files0-from, so that O_CLOEXEC is used properly. This is simpler anyway. * tests/misc/sort-files0-from.pl: Adjust to change in diagnostic wording. build: update gnulib submodule to latest 2017-08-14 Pádraig Brady kill: fix signal number to name lookup on AIX * src/operand2sig.c (operand2sig): AIX uses a different bit pattern in the returned status from the wait() functions and from shells. Therefore hardcode the selection of the lower bits of the number. * NEWS: Mention the fix. build: use the appropriate single file include option with xlc * configure.ac: Set USE_XLC_INCLUDE when __xlc__ is defined. * src/local.mk: Use it to select the appropriate include option. Reported by Michael Felt. tests: avoid false failures on AIX * tests/ln/sf-1.sh: Limit the symlink size to 1MiB to avoid memory exhaustion seen on NFS on AIX, giving: + printf '%0*d' 4294967296 0 + ./tests/ln/sf-1.sh: line 38: printf: warning: 0: Result too large * tests/id/setgid.sh: Skip the test when the adjusted gid would equal 4294967295, as that's reserved on AIX. Reported by Michael Felt. sort: handle musl locale differences in --debug reporting * src/sort.c (main): Don't assume hard_LC_COLLATE implies a successful setting of the locale as musl defaults to UTF8 when failing to set the specified locale. * tests/misc/sort-debug-warn.sh: Adjust for the now separated locale debug info and map the musl specific message back to the common case. Addresses https://bugs.gnu.org/28054 seq: produce consistent error messages upon write error * src/seq.c (io_error): Use the same error message as would be generated at exit time when closing the stdout stream. The inconsistency was added with commit v8.25-26-gc92585b. This was noticed due to an inconsistency in the expected error message generated by seq on musl libc. Addresses https://bugs.gnu.org/28054 tests: fix false failure with large printf formats * tests/misc/printf-surprise.sh: With musl libc the large printf format does succeed, outputting data. To avoid SIGPIPE being generated we ignore that signal and then handle the subsequent EPIPE error. Addresses https://bugs.gnu.org/28054 2017-08-12 Jim Meyering build: adjust warning options to work with latest GCC * configure.ac: Disable some new warnings to avoid false positives. Building with warnings enabled and latest gcc would evoke build failure without these changes. Disable the following in coreutils proper: -Wformat-overflow=2 -Wformat-truncation=2, and disable these for gnulib: -Wformat-truncation=2 -Wduplicated-branches gnulib: update to latest and adjust gl/modules/tempname.diff * gnulib: Update to latest. * gl/modules/tempname.diff: This patch failed to apply. Adjust it to reflect removal of the secure_getenv dependency. chroot: fix typo in preceding change: didn't compile * src/chroot.c (usage): Add backslashes. 2017-08-10 Jim Meyering doc: correct technicality in chroot's --help output * src/chroot.c (usage): Use correct quoting in descriptive diagnostic. We would run `"$SHELL" -i`, not `${SHELL} -i`. 2017-08-09 Assaf Gordon doc: fix join example * doc/coreutils.texi (join invocation): Fix incorrect output in example. Reported by Phlosioneer in https://bugs.gnu.org/28014 . 2017-08-04 Paul Eggert build: update gnulib submodule to latest 2017-08-03 Paul Eggert copy: more-accurate warning about destruction * src/copy.c (copy_internal): * tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh: Say "might destroy", not "would destroy". 2017-08-03 Pádraig Brady maint: avoid a syntax-check failure * src/shred.c (wipename): As per the comment, the arguments to error() are sufficiently quoted, so split the call over multiple lines to avoid the syntax-check. 2017-08-02 Paul Eggert build: update gnulib submodule to latest 2017-08-01 Paul Eggert copy: go back to failing 'cp --backup a~ a' Suggested by Kamil Dudka in: http://lists.gnu.org/archive/html/coreutils/2017-07/msg00072.html * NEWS: Document the changed nature of the fix. * doc/coreutils.texi, tests/cp/backup-is-src.sh: * tests/mv/backup-is-src.sh: Revert previous change. * src/copy.c (source_is_dst_backup): New function. (copy_internal): Use it. Fail instead of falling back on numbered backups when it looks like the backup will overwrite the source. Although this reintroduces a race, it's more compatible with previous behavior. 2017-07-31 Paul Eggert copy: sanity-check --suffix * src/cp.c, src/install.c, src/ln.c, src/mv.c (main): Use set_simple_backup_suffix, to sanity-check the user-supplied backup suffix. copy: make backup files more reliably * NEWS, doc/coreutils.texi (Backup options): Document the change. * bootstrap.conf (gnulib_modules): Add backup-rename. * src/copy.c (copy_internal): Silently switch to numbered backups if a simple backup might lose data. Use backup_file_rename to avoid races with numbered backups. * tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh: Adjust to match new behavior. shred: avoid rename race Use renameat2 to avoid a rename race condition, on recent-enough GNU/Linux. * bootstrap.conf (gnulib_modules): Add renameat2. * src/shred.c: Include renameat2.h. (wipename): Use renameat2 instead of rename. build: update gnulib submodule to latest 2017-07-25 Jim Meyering maint: fix grammar in a shred.c comment * src/shred.c: Remove spurious "to" in an old comment. 2017-07-23 Pádraig Brady maint: fix recent syntax-check failures * .gitignore: Add /lib/utime.h from the recent gnulib update. * src/nproc.c (usage): Adjust spacing to placate help2man. shred: remove redundant zeroing of freed memory * src/shred.c (dopass): shred used to read the input file, and so needed to ensure internal memory was cleared. This is no longer the case since SH-UTILS-1_16f-260-gf381610 so avoid this redundant clearing. (do_wipefd): Likewise. * NEWS: Remove the recent mention of this issue. maint: resync with blake2 upstream * src/blake2/blake2-impl.h: Don't use the equivalent explicit_bzero(). tests: avoid a false failure on AIX * tests/misc/sync.sh: Normalize the error messages when syncing a non read/write directory, as AIX gives the "Is a directory" error. Also ensure that sync(1) returns an error for this case on all systems. 2017-07-20 Paul Eggert shred: use explicit_bzero * NEWS: Document this. * bootstrap.conf (gnulib_modules): Add explicit_bzero. * gl/lib/randint.c (randint_free): * gl/lib/randread.c (randread_free): * src/blake2/blake2-impl.h (secure_zero_memory): * src/shred.c (dopass, do_wipefd): Prefer explicit_bzero to memset when erasing secrets. build: update gnulib submodule to latest 2017-07-10 Andreas Schwab nproc: fix indentation of usage output * src/nproc.c (usage): Align output. 2017-07-10 Jim Meyering groups: do not exit early Most programs take care to operate on all command-line-specified operands before exiting. That is an important feature that allows to identify all problems with the first run. However, groups would exit upon the first problematic user name. Bug introduced via commit v6.10-56-g167b8025ac. * src/groups.c (main): Do not exit immediately upon error. * tests/misc/groups-process-all.sh: New file. Test for this. * tests/local.mk (all_tests): Add it. * NEWS (Bug fixes): Mention this. 2017-07-08 Jim Meyering tests: groups-dash.sh: avoid false failure * tests/misc/groups-dash.sh: Avoid false failure on a system for which "none" is a valid user name. The first invocation would succeed, and the second would fail with "groups: ‘--’: no such user". Use a user name that cannot exist. doc: tweak wording * NEWS (Bug fixes): Tweak wording of the mv/cp-vs-symlink-ownership entry and the one about df. 2017-06-28 Assaf Gordon expr: add multibyte support Discussed in https://bugs.gnu.org/26779 . * NEWS: Mention the improvement. * bootstrap.conf: Add gnulib modules mbslen,mbschr. * src/expr.c (mbs_logical_substr): New function to return a substring based on logical character positions (instead of bytes). (mbs_logical_cspn): Similar to strcspn/mbscspn, but returns number of logical characters instead of byte offset. (mbs_offset_to_chars): New function to return number of logical characters fitting in a given byte offset. (docolon): Report matched logical characters instead of bytes. (eval6): For length/substr/index operations, use logical characters instead of bytes by calling the above new functions. * tests/misc/expr.pl: Repeat all tests with non-C locale to detect any regressions. * tests/misc/expr-multibyte.pl: New tests with multibyte input. * tests/local.mk: Add new test file. 2017-06-21 Jim Meyering maint: avoid spurious "make distcheck" failure When the generated file, doc/constants.texi, happens to be older than doc/coreutils.info, it will not be updated until/unless its generated contents change. This is due to way that rule is careful to update the file, to avoid provoking a pointless rerunning of makeinfo. Note that this does not happen when one first runs "make distclean", as recommended in README-release. However, I sometimes run it as a more-rigorous "make check", and shouldn't have to manually run "make distclean" first, in that case. Before this change, one could reproduce the failure by running `touch -dyesterday doc/constants.texi && make distcheck`. It would fail with "makeinfo: could not open ../../doc/coreutils.info-t for writing: Permission denied" * Makefile.am (dist-hook): Touch the two generated files, so that they cannot be out of date wrt doc/coreutils.texi. 2017-06-17 Pádraig Brady maint: use C99 for loop initial declarations where possible This results in a net reduction of about 120 lines. tail: only use inotify with regular files * src/tail.c (any_non_regular): A new function to check passed files. (main): Use the above to skip inotify if any non regular files passed like /dev/tty or /dev/ttyUSB0 etc. * tests/tail-2/inotify-only-regular.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/21265 and http://bugs.gnu.org/27368 tail: with -f don't warn if doing a blocking read of a tty * src/tail.c: (main): Only issue the warning about -f being ineffective when we're not going into simple blocking mode. * tests/tail-2/follow-stdin.sh: Ensure the warning is output correctly. Fixes http://bugs.gnu.org/27368 2017-06-11 Pádraig Brady tail: exit promptly when output no longer writable This will support use cases like: tail -f file.log | grep -q trigger && process_immediately * src/tail.c (check_output_alive): A new function that uses select on fifos or pipes to detect if they're broken. (tail_forever): Call check_output_alive() periodically. (tail_forever_inotify): Merge the select() call from check_output_alive() into the select() originally present for the --pid case, and adjust accordingly. * tests/tail-2/pipe-f.sh: Add test cases. * NEWS: Mention the improvement. 2017-06-11 Jim Meyering maint: update to work with GCC7's -Werror=implicit-fallthrough=5 * src/system.h (FALLTHROUGH): Define. * src/cp.c (main): Use new FALLTHROUGH macro in place of comments. * src/basename.c (main): Likewise. * src/dircolors.c (append_quoted): Likewise. * src/echo.c (main): Likewise. * src/fold.c (main): Likewise. * src/join.c (main): Likewise. * src/kill.c (main): Likewise. * src/ls.c (get_funky_string, gobble_file): Likewise. * src/sort.c (parse_field_count, main): Likewise. * src/stat.c (print_it): Likewise. * src/tail.c (parse_obsolete_option): Likewise. * src/test.c (posixtest): Likewise. * src/wc.c (wc): Likewise. * src/who.c (main): Likewise. 2017-06-07 Pádraig Brady tail: with --pid, ensure all inotify events are processed * NEWS: Mention the bug fix. * src/tail.c (tail_forever_inotify): With --pid, avoid waiting for new events if there are still events to process. * tests/tail-2/inotify-dir-recreate.sh: Adjust to trigger. tests: fix issues with recently added tail test * tests/tail-2/inotify-dir-recreate.sh: Skip when inotify is not usable. Also remove a bash specific &> construct. 2017-06-03 Pádraig Brady copy: don't fail when unable to chown symlinks * src/copy.c (copy_internal): Honor the x->require_preserve flag for symlinks as we do for ordinary files, so we don't exit with failure upon failure to chown a symbolic link. * NEWS: Mention the bug fix. 2017-05-29 Sebastian Kisela doc: mention `setpriv --no-new-privs` feature in runcon info * doc/coreutils.texi (runcon invocation): Mention setpriv usage. Discussed at https://bugzilla.redhat.com/1360903 2017-05-18 Pádraig Brady mv: distinguish copy and rename operations with --verbose * src/copy.c (copy_internal): In x->move_mode distinguish whether we're copying, creating directory, or renaming. * tests/mv/backup-dir.sh: Adjust to new output. * tests/mv/mv-n.sh: Likewise. * tests/mv/mv-special-1.sh: Likewise. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/26971 2017-05-11 Prateek saxena uptime: remove inconsistent AM/PM from current time * src/uptime.c (main): 00-23 was always used for the hour component of the current time, so remove the AM/PM output (which was only present in some locales anyway). Also add seconds to the time to be more consistent with the usual procps-ng uptime implementation on GNU/Linux. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/26783 2017-05-04 Pádraig Brady maint: fix various typos in recent commits * NEWS: Grammar fixes. * HACKING: Likewise. 2017-05-04 Jaak Ristioja doc: Fixed typo in timeout man page * man/timeout.x: Correct spelling of "currently". Fixes http://bugs.gnu.org/26762 2017-04-30 Pádraig Brady doc: update the instructions for generating a coverage report * HACKING: Change from explicit instructions to using gnulib provided coverage testing targets. Also include instructions for adding root only tests to the report. Fixes http://bugs.gnu.org/26709 2017-04-27 Paul Eggert dd: simplify translator’s jobs * src/dd.c (print_xfer_stats): Format the SI units directly, without translating them, to simplify the translators’ jobs. See Bug#26621. 2017-04-27 Pádraig Brady date,touch: test and document large TZ security issue Add a test for CVE-2017-7476 which was fixed in gnulib at: http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=94e01571 * tests/misc/date-tz.sh: Add a new test which overwrites enough of the heap to trigger a segfault, even without ASAN enabled. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. 2017-04-27 Pádraig Brady build: update gnulib submodule to latest * .gitignore: Add new entry as indicated by `make syntax-check`. 2017-04-24 Paul Eggert dd: status=progress outputs "6 s", not "6.00001 s" Problem reported by Benno Schulenberg (Bug#26621). * NEWS: Document this. * src/dd.c (print_xfer_stats): With status=progress, format times with %.0f rather than %g. Improve translator comments. 2017-04-22 Paul Eggert build: update gnulib submodule to latest maint: remove unused functions and constants These were found by clang. * gl/lib/rand-isaac.c (min): * gl/lib/randint.c (shift_right): * src/md5sum.c (algorithm): Remove; unused. date: adjust to gnulib parse-datetime changes * doc/coreutils.texi (Options for date): Capitalize a sentence. * tests/misc/date-debug.sh: Adjust --debug output to match recent changes to Gnulib’s parse-datetime module. build: update gnulib submodule to latest * gl/modules/tempname.diff: Update to match current Gnulib. 2017-04-18 Bogdan Drozdowski shred: fix invalid pattern generation for certain sizes * src/shred.c (fillpattern): Fix the "off by one" issue when testing whether we have enough space to copy the already written portion of the buffer to the remainder of the buffer. Specifically for buffer sizes that are (3*(2^x))+1, i.e. 7,13,... we both use an uninitialized byte and invoke undefined behavior in memcpy() operation on overlapping memory regions. * tests/misc/shred-passes.sh: Add an invocation that will trigger either valgrind UMR, or ASAN like: ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges #1 0x403065 in fillpattern src/shred.c:293 A direct test is awkward due to the random writes surrounding the problematic pattern writes. Fixes http://bugs.gnu.org/26545 2017-04-17 Bo Rydberg doc: fix awk example for getting penultimate field * doc/coreutils.texi (cut invocation): Add required brackets. Fixes http://bugs.gnu.org/26519 2017-04-06 Sebastian Kisela tail: revert to polling if a followed directory is replaced * src/tail.c (tail_forever_inotify): Add the IN_DELETE_SELF flag when creating watch for the parent directory. After the parent directory is removed, an event is caught and then we switch from inotify to polling mode. Till now, inotify has always frozen because it waited for an event from a watched dir, which has been already deleted and was not added again. * tests/tail-2/inotify-dir-recreate.sh: Add a test case. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/26363 Reported at https://bugzilla.redhat.com/1283760 2017-04-06 Pádraig Brady maint: fix syntax-check issues in previous tty commit * src/tty.c: Avoid EXIT_FAILURE to be more descriptive and to placate sc_some_programs_must_avoid_exit_failure. 2017-04-05 Paul Eggert tty: handle misconfigured namespaces On some platforms, isatty succeeds but ttyname fails. POSIX does not seem to allow this, but there it is. Problem reported by Christian Brauner (Bug#26371). While we’re at it, check for errors more carefully and return a new exit status 4 if stdin is closed or a similar error occurs. * doc/coreutils.texi (tty invocation): Document new behavior. * init.cfg (stderr_fileno_): Don't assume have_input_tty is not in the environment. * src/tty.c (TTY_STDIN_ERROR): New constant. (main): Exit with nonzero status if there is a usage error, like other coreutils programs. Check for error in getting stdin type. * tests/misc/tty.sh: New file. * tests/local.mk (all_tests): Add it. 2017-04-03 Pádraig Brady doc: refactor and update expand and unexpand --help * src/expand-common.c (emit_tab_list_info): A new function to output the extended info on --tab=LIST, including the new '+' and '/' prefixes. * src/expand-common.h: Declare the above. * src/expand.c (usage:): Call emit_tab_list_info and match alignment with that used in unexpand --help. * src/unexpand.c (usage): Likewise. 2017-04-03 Jacob Keller expand,unexpand: add support for incremental tab stops Support --tabs="1,+8" which is equivalent to --tabs="1,9,17,..." useful for viewing unified diff output with its 1 character gutter for example. * doc/coreutils.texi ({expand,unexpand} invocation): Document, using diff processing as the example. * src/expand-common.c (set_increment_size): Update the new increment_size global. (parse_tab_stops): Handle the new '+' prefix. (finalize_tab_stops): Verify both '+' and '/' prefixes are not used together. * tests/misc/expand.pl: Add test cases. * NEWS: Mention the new feature. 2017-03-30 Paul Eggert sort: update comment * src/sort.c: Update identifiers in comment. 2017-03-30 Chris Davies doc: clarify in dd man page that bs= overrides [io]bs= * src/dd.c (usage): Add the extra info. Reported in https://bugs.debian.org/859021 2017-03-28 Ludovic Courtès tests: avoid false ulimit failure on some systems * tests/misc/cut-huge-range.sh: On some systems returns_ may use more memory, so incorporate that in the determination of the ulimit value to use. Noticed on ARMv7 with bash-4.4.12, and x86_64 with bash-4.2.37. Fixes http://bugs.gnu.org/26253 2017-03-28 Pádraig Brady maint: avoid syntax check failure with wrapped returns_ * cfg.mk (sc_prohibit_env_returns): Allow wrapped calls to return_ of the form: `wrapper_ returns_ ...` which is needed with the following commit. 2017-03-28 Michael Heimpold split: add new --hex-suffixes option * doc/coreutils.texi (split invocation): Document the new option. * src/split.c (usage): Likewise. (main): Process the new option much like --numeric-suffixes, but with an adjusted alphabet. * tests/split/numeric.sh: Refactor to support --hex mode. * NEWS: Mention the new feature. 2017-03-28 Pádraig Brady md5sum,b2sum,sha*sum: don't erroneously trigger BSD reversed mode * src/md5sum.c (split_3): Verify hex digits internally before triggering the global bsd_reversed mode flag. (bsd_split_3): Likewise. * tests/misc/md5sum-bsd.sh: Add a test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/26263 2017-03-27 Philipp Thomas df: avoid querying excluded file systems * src/df.c (filter_mount_list): Avoid stat() on explicitly excluded file systems, which is especially significant in cases like `-x nfs` which may hang. * NEWS: Mention the bug fix. 2017-03-26 Pádraig Brady maint: avoid a static analysis warning in expand-common * src/expand-common.c (next_file): We're dependent on calling this function with NULL to initialize things appropriately. So enforce this with assert(), which avoids a warning from clang-anaylzer. split: process more efficiently when filters exit early * src/split.c (bytes_split): Don't write to an existing filter if it has exited. When filters exit early, skip input data if possible. Refactor out 2 redundant variables. * tests/split/filter.sh: Improve test coverage given the new more efficient processing. Also use a 10TB file to expand the file systems tested on. 2017-03-26 Pádraig Brady split: ensure input is processed when filters exit early commit v8.25-4-g62e7af0 introduced the issue as it broke out of the processing loop irrespective of the value of new_file_flag which was used to indicate a finite number of filters or not. For example, this ran forever (as it should): $ yes | split --filter="head -c1 >/dev/null" -b 1000 However this exited immediately due to EPIPE being propagated back through cwrite and the loop not considering new filters: $ yes | split --filter="head -c1 >/dev/null" -b 100000 Similarly processing would exit early for a bounded number of output files, resulting in empty data sent to all but the first: $ truncate -s10T big.in $ split --filter='head -c1 >$FILE' -n 2 big.in $ echo $(stat -c%s x??) 1 0 I was alerted to this code by clang-analyzer, which indicated dead assigments, which is often an indication of code that hasn't considered all cases. * src/split.c (bytes_split): Change the last condition in the processing loop to also consider the number of files before breaking out of the processing loop. * tests/split/filter.sh: Add a test case. * NEWS: Mention the bug fix. 2017-03-11 Pádraig Brady tests: avoid a false failure on OS X 10.5.8 * tests/misc/sort-debug-keys.sh: Disparate LC_CTYPE and LC_MESSAGES are not supported, with the result LC_MESSAGES=C is used throughout. Therefore just set LC_ALL in the test, and normalize the message variants with sed. Reported and tested by J Rogowsky. build: fix missing renameat() on OS X 10.5.8 * bootstrap.conf: Depend on renameat. Reported and tested by J Rogowsky. Fixes http://bugs.gnu.org/26044 2017-03-10 Paul Eggert tests: port to tzdb-2017a Problem reported by Bernhard Voelker in: http://lists.gnu.org/archive/html/coreutils/2017-03/msg00026.html * tests/misc/date-debug.sh: Port test to tzdb 2017a, and future-proof the America/Belize test. 2017-03-09 Pádraig Brady build: for factor use C in more cases for arm64 and ppc64 * src/longlong.h: Sync from gmp repo incorporating: Use asm-free umul_ppmm() on arm64 and ppc64. doc: rearrange a recent bug entry to an improvement in NEWS * NEWS: The stat,tail change was an improvement, not a bug fix. * cfg.mk [old_NEWS_hash]: update with `make update-NEWS-hash`. maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.27 * NEWS: Record release date. build: update gnulib submodule to latest Removes extraneous parse-datetime.c generated during build stat,tail: support "RDT" Linux kernel control file system * src/stat.c (human_fstype): This file system is the user interface for resource allocation in Intel's Resource Director Technology. doc: spelling fix for recent doc addition * doc/coreutils.texi (join invocation): s/preceeding/preceding/. 2017-03-08 Bernhard Voelker build: avoid redundant build of tr with --enable-single-binary * src/local.mk [check-duplicate-no-install]: Depend on the single-binary tr, or the system tr, as the edge case where these are not available only result in the sanity check being effectively ignored. 2017-03-08 Assaf Gordon build: fix 'install-html' target Switching to non-recursive makefiles broke the 'install-html' target: The gettext plumbing requires an 'install-html' target in po/Makefile. This was fixed in gettext v0.19.8.1-41-ge5a008a, but packages using older gettext need to manually patch po/Makefile.in.in. Reported (for 'sed') and suggested fix by Eric Blake in https://bugs.gnu.org/25690 . * bootstrap.conf (bootstrap_epilogue): Add 'install-{html,pdf,dvi,ps}' targets to po/Makefile.in.in (if needed). 2017-03-04 Pádraig Brady maint: bump makeinfo --version requirement to 6.1 * bootstrap.conf: s/4.13/6.1/ as versions previous to that generated invalid html with interspersed tags that were visible to the user. Version 6.1 is available for a year now, and is available in most distros. 2017-03-04 Bernhard Voelker doc: avoid makeinfo warning makeinfo issues the following: doc/coreutils.texi:6568: warning: @sc argument all uppercase,\ thus no effect. * doc/coreutils.texi (join invocation): Remove the @sc macro around the all uppercase "GNU". 2017-03-03 Pádraig Brady build: update gnulib submodule to latest Fixes a test-lock failure on MacOS timeout: handle multiple children on solaris * src/timeout.c (install_sigchld): A new function to install the SIGCHLD handler using sigaction() rather than signal(), because with the latter on solaris the signal handler is reset to default and thus sigsuspend() only returns for the first finished child. Reported by Assaf Gordon. tests: avoid a spurious failure on older debian * tests/misc/cut-huge-range.sh: Bump up the ulimit, to avoid a false failure due hitting the previously detected ulimit. Reported by Assaf Gordon. build: fix libstdbuf build on AIX 7 * src/libstdbuf.c: undef malloc so as libstdbuf is not linked with gnulib, and anyway the replacement is never needed since we never malloc(0). Reported by Assaf Gordon. 2017-03-02 Assaf Gordon doc: expand 'join' info section * doc/coreutils.texi (join invocation): Expand section to add examples and more details. Suggested by Dan Jacobson in https://bugs.gnu.org/25870 2017-03-01 Pádraig Brady doc: give a stronger security warning in md5/sha1 man pages * man/md5sum.x: Give a more direct warning againt the use of this hash algorithm for security purposes. * man/sha1sum.x: Likewise. Suggested by Jim Meyering. expand: avoid an extraneous warning on 32 bit * src/expand-common (parse-tab-stops): Exit earlier upon overflow so another warning isn't issued (on 32 bit) in add_tab_stop(). Flagged in https://hydra.nixos.org/build/49499970 doc: indicate sha1 has the same limitations as md5 * doc/coreutils.texi (sha1sum invocation): Given that a SHA-1 preimage attack has occurred as documented at http://shattered.io/, document sha1sum as having the same limitations as md5sum. (md5sum): Parameterize the warning for use in both cases. * man/md5sum.x: Mention b2sum(1) as a more secure alternative. * man/sha1sum.x: Give the same warning as done for md5sum(1). maint: avoid a -Werror=null-dereference with GCC-6.3.1 * src/stty.c (sane_mode): Assert to inform the compiler we know the pointer will be valid. expand,unexpand: support specifying a trailing tab size * doc/coreutils.texi (expand invocation): Document the feature. (unexpand invocation): Likewise. * src/expand-common.c (extend_size): A new global to use when the last tab stop is prefixed by '/'. (set_extend_size): A new function to validate and set the new extend_size global. (parse_tab_stops): Call set_extend_size() for '/' prefixes. (finalize_tab_stops): Ensure a single specified '/' is treated like a standard tabsize, but also ensure that when '/' is specified with a single other entry that we process as a list rather than a tab size. (get_next_tab_stop): Use the tab size if set, for items after the user specified tab position list. * tests/misc/expand.pl: Add test cases * NEWS: Mention the new feature. Fixes http://bugs.gnu.org/25540 2017-02-26 Pádraig Brady nproc: support OMP_THREAD_LIMIT to set a max value This comes from the latest gnulib. Also handling of OMP_NUM_THREADS has been adjusted to support comma separated values indicating a nesting level, in which case the first value is taken. Also OMP_NUM_THREADS=0 is now ignored instead of being treated as 1, to match the behavior of libgomp. * NEWS: Mention the OMP_THREAD_LIMIT improvement, and OMP_NUM_THREADS now handling nested values. * doc/coreutils.texi (nproc invocation): Describe OMP_THREAD_LIMIT as a way to set the max value, with OMP_THREAD_LIMIT setting the min. * tests/misc/nproc-override.sh: A new test to exercise the updated gnulib code with all combinations of these OMP variables. * tests/local.mk: Reference the new test. 2017-02-26 Pádraig Brady build: update gnulib submodule to latest 2017-02-23 Pádraig Brady doc: add NEWS for an improvement to dd in the last release * NEWS: Mention the avoidance of the gotcha with specifying a hex constant like count=0x1000 etc. as that previously was silently interpreted as 0. 2017-02-23 Pádraig Brady cp: set SELinux context for --parents directories * src/copy.c (set_process_security_ctx, set_file_security_ctx): Export for use in cp.c. * src/copy.h: Likewise. * src/cp.c (make_dir_parents_private): Call the exported functions to set the security context for new and updated directories. * tests/cp/cp-a-selinux.sh: Add a test case. Fixes http://bugs.gnu.org/25378 2017-02-18 Pádraig Brady maint: tweaks so syntax tests pass for previous commit * .gitignore: placate sc_gitignore_redundant. * po/POTFILES.in: placate sc_po_check. 2017-02-16 Bernhard Voelker doc: avoid makeinfo warning Commit v8.26-38-g99deaff introduced this warning: MAKEINFO doc/coreutils.info ./doc/coreutils.texi:10268: \ warning: `.' or `,' must follow @xref, not `@'. * doc/coreutils.texi (readlink invocation): Add '.' after @xref macro. 2017-02-16 Paul Eggert maint: xsetmode renamed to xbinary-io * bootstrap.conf, src/base64.c, src/cat.c, src/cksum.c: * src/head.c, src/md5sum.c, src/od.c, src/split.c, src/sum.c: * src/tac.c, src/tail.c, src/tee.c, src/tr.c, src/wc.c: Adjust to renaming of the xsetmode module to xbinary-io, and of the xsetmode function to xset_binary_mode. build: update gnulib submodule to latest 2017-02-15 Paul Eggert maint: use xsetmode, not xfreopen This fixes a bug noted by Eric Blake. Code was using xfreopen to change files to binary mode, but this fails for stdout when in append mode. Such code should use xsetmode instead. This affects only the port on platforms like MS-Windows which distiguish text from binary I/O. * bootstrap.conf (gnulib_modules): Remove xfreopen and add xsetmode. Sort. * src/base64.c (main): * src/cat.c (main): * src/cksum.c (cksum): * src/head.c (head_file, main): * src/md5sum.c (digest_file): * src/od.c (open_next_file): * src/split.c (main): * src/sum.c (bsd_sum_file, sysv_sum_file): * src/tac.c (tac_file, main): * src/tail.c (tail_file): * src/tee.c (tee_files): * src/tr.c (main): * src/wc.c (wc_file): Use xsetmode, not xfreopen. build: update gnulib submodule to latest 2017-02-13 Pádraig Brady maint: tweaks so syntax tests pass for previous commit * src/force-link.h: Don't include headers already included by system.h * src/force-link.c: Likewise. Also include system.h and explicitly mark extern functions as such. 2017-02-12 Paul Eggert ln: replace destination links more atomically If the file B already exists, commands like 'ln -f A B' and 'cp -fl A B' no longer remove B before creating the new link. Instead, they arrange for the new link to replace B atomically. This should fix a race condition reported by Mike Crowe (Bug#25680). * NEWS, doc/coreutils.texi (cp invocation, ln invocation): Document this. * bootstrap.conf (gnulib_modules): Add symlinkat. * src/copy.c, src/ln.c: Include force-link.h. * src/copy.c (same_file_ok): It's also OK to remove a destination symlink when creating symbolic links, or when the source and destination are on the same file system and when creating hard links. * src/copy.c (create_hard_link, copy_internal): * src/ln.c (do_link): Rewrite using force_linkat and force_symlinkat, to close a window where the destination temporarily does not exist. * src/cp.c (main): Do not set x.unlink_dest_before_opening merely because we are in link-creation mode. * src/force-link.c, src/force-link.h: New files. * src/local.mk (copy_sources, src_ln_SOURCES): Add them. * tests/cp/same-file.sh: Adjust test case to match fixed behavior. 2017-02-10 Tobias Stoeckmann timeout: fix race possibly terminating wrong process The race is unlikely, as timeout(1) needs to receive a signal in the few operations between waitpid() returning and exit(). Also the system needs to have reallocated the just released pid in this time window. Previously we never disabled the signal handler that sent the termination signal to the "child" pid. However once waitpid() has reaped the child, the system is free to allocate that pid, so we must ensure we don't process any further signals. * build-aux/gen-lists-of-programs.sh: Build timeout(1) optionally... * configure.ac: ...predicated on sigsuspend() being available. * src/timeout.c (block_cleanup): A new function to ensure the cleanup() handler is disabled after waitpid has returned. (main): Use sigsuspend() to wait with cleanup() enabled but disabled once it returns, and thus disabled for the waitpid() call. (monitored_pid): Change to the more accurate pid_t. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/25624 2017-02-10 Pádraig Brady doc: note the relationship between realpath and readlink * doc/coreutils.texi (realpath invocation): Mention that realpath is the preferred command for canonicalization. (readlink invocation): Likewise. * man/readlink.x: Likewise. 2017-02-08 Janne Snabb tail: fix output of redundant headers when resuming * src/tail.c (check_fspec): Only enable printing of the file header if we've actually read some data and this is a new file. Also move printing of the file header to... (dump_remainder): ...here, to allow printing only when data read. * tests/tail-2/overlay-headers.sh: A new test for suspension and resumption of tail. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/23539 2017-02-08 Pádraig Brady tests: fix tail test race causing false failure * tests/tail-2/retry.sh: The replacement of the "missing" directory is not atomic, and therefore tail(1) can take a different path, especially if there is a delay between the rmdir(2) and creat(2). This is noticeable for example with `make coverage` because in that case the coverage files written by rmdir(1) on exit, induce a significant delay thus triggering the issue. tail: fix erroneous status about 'giving up' on file * src/tail.c (recheck): Set f->ignore before we use it to show the appropriate error. * tests/tail-2/retry.sh: Ensure the "giving up" message is not presented. 2017-02-08 Pádraig Brady doc: only distribute 5 years of ChangeLogs Remove old log files that have corresponding entries in the source code repository. This saves about 2.5MB uncompressed, 0.5M compressed. * Makefile.am (gen-ChangeLog): Adjust to taking all logs since a particular version (8.15 in this case). Also mention in the truncated log where to get older entries. (changelog_etc): Remove the no longer distributed files. * build-aux/git-log-fix: Remove now unused entries. * ChangeLog-200[5-8]: Delete. * doc/ChangeLog-2007: Likewise. * po/ChangeLog-2007: Likewise. * old/*: Likewise. 2017-02-07 Maxime de Roucy maint: fix HACKING instructions to run a single test * HACKING: s/make TEST=/make check TEST=/ The 'check' target was missing there since v8.20-57-geac397e. 2017-01-25 Pádraig Brady build: fix issue with HAVE_FALLOCATE on centos5 * src/copy.c (punch_hole): Work around an empty definition of HAVE_FALLOCATE which leads to a build error of: "error: #if with no expression" That was triggered by the inclusion of in commit v8.25-68-g89e1fef with kernel-headers-2.6.18. Reported by Nelson H. F. Beebe 2017-01-25 Manolis Ragkousis (tiny change) build: Properly expand cu_install_program when cross-compiling * src/local.mk (cu_install_program): Replace @INSTALL_PROGRAM@ with @INSTALL@ when cross-compiling; missed in commit 477a1e8e. Message-Id: <20170125163329.5690-1-manolis837@gmail.com> 2017-01-21 Mike Swanson dircolors: highlight windows archive format * src/dircolors.hin: Match *.{wim,swm,dwn,esd} 2017-01-21 Pádraig Brady maint: mention the recent date time zone bug fix * NEWS: Add the bug fix from commit v8.26-27-gb14be50 2017-01-21 Assaf Gordon maint: appease syntax-check failures due to recent updates * bootstrap: s/time stamp/timestamp/. * old/fileutils/NEWS: Likewise. * src/tail.c: Avoided a long line. 2017-01-21 Paul Eggert date: fix TZ= regression Problem reported by Paul Wise for Debian, in: https://bugs.debian.org/851934 This is fallout from the fix for GNU Bug#23035. * src/date.c (batch_convert): New args TZ and TZSTRING. All uses changed. (batch_convert, main): Adjust to parse_datetime2 API change. (main): Allocate time zone object. * tests/misc/date-debug.sh: Fix incorrect test case, caught by the fix. * tests/misc/date.pl: Test the fix. build: update gnulib submodule to latest 2017-01-15 Jim Meyering maint: update README-hacking, now that vc-dwim accepts --init * README-hacking: I've just released vc-dwim-1.8, so we can improve the documentation to reference its --initialize option here. 2017-01-15 Paul Eggert date: output "-00" for indeterminate time zone * NEWS: Document this behavior, which comes with recent Gnulib. * doc/coreutils.texi (Formatting file timestamps, du invocation) (Time conversion specifiers, Setting the time, Options for date): Mention when -00 is output for numeric time zones. Be more careful about Internet RFC numbers, ISO 8601, etc. date: new option spelling --rfc-email * NEWS: * doc/coreutils.texi (Time conversion specifiers) (Options for date, Examples of date): Document this. * src/date.c (rfc_email_format): Rename from rfc_2822_format. All uses changed. (usage, long_options): Support --rfc-email. maint: modernize URLs A lot of this is converting http: to https:. Also, gmane went away, so remove URLs that no longer work and are not easy to figure out what they were. Some of this stuff is so old that it no longer matters anyway. build: update gnulib submodule to latest 2017-01-10 Bernhard Voelker doc: move "File timestamps" to a separate chapter The above new section looked a bit odd as the only general documentation in between the utility chapters. * doc/coreutils.texi (File timestamps): Move to a separate chapter. 2017-01-10 Eric Blake maint: fix recent syntax check failures Commit 4f650aad was incomplete; it changed NEWS but not the hash, and introduced a grammar error. * cfg.mk (old_NEWS_hash): Update via 'make update-NEWS-hash'. * doc/coreutils.texi (File timestamps): Fix doubled word. 2017-01-09 Paul Eggert doc: cover file timestamps better Prompted by a bug report from Scott Deerwester (Bug#25407). * doc/coreutils.texi (File timestamps): New section. Revamp other sections to use this new section, and use more-consistent terminology. maint: standardize on "timestamp" as per POSIX 2017-01-09 Pádraig Brady stty: ensure no side effects from invalid options * src/stty.c (apply_settings): A new function refactored from main() that is used to both check and apply options. (main): Call apply_settings before we open the device, so all validation is done before interacting with a device. * NEWS: Mention the improvement. * tests/misc/stty.sh: Add a test case. 2017-01-06 Assaf Gordon tests: improve 'date --debug' tests Update tests following improvements to gnulib's parse-datetime.y module. See https://lists.gnu.org/archive/html/bug-gnulib/2017-01/msg00002.html * tests/misc/date-debug.sh: Add tests for each of the gnulib changes. 2017-01-06 Assaf Gordon build: update gnulib submodule to latest 2017-01-01 Pádraig Brady maint: update all copyright year number ranges Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year. 2016-12-28 Zooko doc: recommend b2sum as well as SHA2 b2sum is faster, easier to use safely, and more future-proof 2016-12-28 Pádraig Brady tests: avoid false fails on NFS due to EPERM * tests/chgrp/basic.sh: On some NFS setups a user is not allowed to set a group on a file even if a member of that group. Therefore skip this test on remote file systems. * tests/chgrp/default-no-deref.sh: Likewise. * tests/chgrp/no-x.sh: Likewise. * tests/chgrp/posix-H.sh: Likewise. * tests/chgrp/recurse.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: Change to skipping on remote file systems in the standard way. 2016-12-27 Paul Eggert doc: Update POSIX part of README (Bug#25259) 2016-12-26 Pádraig Brady wc: with only --bytes, determine size more efficiently * src/wc.c (wc): Avoid reading the end of the file when the size is not a multiple of PAGE_SIZE, as the special case handling for files in /proc and /sys is only required when st_size is 0 or a multiple of PAGE_SIZE. * tests/misc/wc-proc.sh: Add a test case. 2016-12-20 Pádraig Brady maint: correct the version for the previous bug fix While st_size would have been incorrect for subsequent files since v7.1, it was only used since v8.24. * tests/misc/wc-files0.sh: s/7.1/8.24/ * NEWS: Likewise. Reported by Bernhard Voelker 2016-12-19 William R. Fraser wc: fix wrong byte counts when using --files-from0 * src/wc.c (main): Reset fstatus[0].failed between files when reusing the fstatus[0] entry in --files-from0 mode. This ensures a stat() is done for each file, avoiding incorrect counts and redundant reading. * NEWS: Mention the bug fix. * tests/misc/wc-files0.sh: Add a test case. Fixes http://bugs.gnu.org/23073 2016-12-18 Pádraig Brady tests: fix typos in previous commit * init.cfg (skip_if_mcstransd_is_running_): Fix typos _introduced my me_ in the previous commit. 2016-12-18 Nicolas Iooss tests: support non-MLS enabled SELinux systems When running "make check" on a Linux system running SELinux with a non-MLS policy, tests/mkdir/restorecon.sh test fails with: chcon: invalid context: root:object_r:tmp_t:s0: Invalid argument Indeed in such a configuration, contexts cannot have ":s0" suffix. * init.cfg (get_selinux_type): Refactor this function to here from various tests. Update to work with a non-MLS policy. (mls_enabled_): A new function to detect if MLS is enabled. (skip_if_mcstransd_is_running_): Update to not skip when MLS is not enabled. * tests/mkdir/restorecon.sh: Use a valid non-MLS context when needed. * tests/install/install-Z-selinux.sh: Likewise. * tests/cp/cp-a-selinux.sh: Likewise. * tests/misc/selinux.sh: Likewise. * tests/misc/chcon.sh: Skip if non-MLS as --range used throughout. Fixes http://bugs.gnu.org/22631 2016-12-08 Torbjörn Granlund factor: retry properly if Pollard rho gives a trivial factorization * src/factor.c (factor_using_pollard_rho): Handle trivial factor g = n. (factor_using_pollard_rho2): Handle trivial factor g1 = n1, g0 = n0. * tests/misc/factor.pl: Add a test case. Fixes http://bugs.gnu.org/25135 2016-12-08 Niels Möller factor: fix infinite loop in gcd2_odd * src/factor.c (gcd2_odd): Fix the case a1 == 0, a0 == 0. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/25135 2016-12-02 Pádraig Brady doc: fix --help for: od -t f[SIZE] * src/od.c (usage): SIZE is that of float, not integer. 2016-11-30 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.26 * NEWS: Record release date. tests: fix false failure with spaces in $PWD * tests/misc/ptx-overrun.sh: Quote appropriately to avoid this recently added issue, noticed by `make taint-distcheck`. maint: avoid "make distcheck" failure without excess .deps directories * Makefile.am (my-distcheck): Add the -r option to xargs so that rmdir doesn't return an error when there are no extraneous .deps dirs. 2016-11-29 Pádraig Brady tests: fix ERRORs and false FAILs on some platforms * tests/misc/ls-time.sh: Skip the test rather than ERROR when `touch -m -d ...` fails (Hurd). * tests/tail-2/follow-stdin.sh: Avoid false FAILs by ignoring the variances in sterror output. * tests/rm/rm-readdir-fail.sh: Likewise. Also avoid ERRORs on systems that don't define _D_EXACT_NAMELEN. 2016-11-28 Pádraig Brady head: fix processing of non-seekable input as seekable * src/head.c (elide_tail_bytes_file): Ensure we don't use st_size unless we've previously used seek() to determine the CURRENT_POS in the seekable file. This was seen to cause issue on FreeBSD 11 when the pipe buffer was filled with `yes | head --lines=-0`, in which case st_size was 64KiB while ST_BLKSIZE() was 4KiB. Reported by Assaf Gordon. 2016-11-28 Kamil Dudka install,mkdir: fix handling of -DZ and -pZ, respectively ... in the case where two or more directories nested in each other are created and each of them defaults to a different SELinux context. * src/install.c (make_ancestor): When calling defaultcon(), give it the same path that is given to mkdir(). The other path is not always valid wrt. current working directory. * src/mkdir.c (make_ancestor): Likewise. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1398913 2016-11-28 Pádraig Brady tac: fix mem corruption when failing to read non seekable inputs This was detected with ASAN, but can also be seen without ASAN with: $ tac - - <&- tac: standard input: read error: Bad file descriptor *** Error in `tac': malloc(): memory corruption: 0x... * src/tac.c (copy_to_temp): Don't close our output stream on (possibly transient) output error, or on input error. (temp_stream): clearerr() on the stream about to be reused, to ensure future stream use is not impacted by transient errors. * tests/misc/tac-2-nonseekable.sh: Add a test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/25041 2016-11-27 Pádraig Brady tail: fix uninitialized memory read when failing to read file Reproduced under UBSAN with `tail -f <&-` giving: tail.c:2220:18: runtime error: load of value 190, which is not a valid value for type ‘_Bool' * src/tail.c (tail_file): Ensure f->ignore is initialized in all cases where we can't tail the specified file. * tests/tail-2/follow-stdin.sh: Add a test case which checks stderr has no UBSAN warnings. Fixes http://bugs.gnu.org/25041 2016-11-27 Pádraig Brady doc: add NEWS entries for recent changes * NEWS: Mention in improvements about the workaround for the glibc issue with closed stdin, and the new supported file systems. * tests/misc/b2sum.sh: Spelling fix. 2016-11-27 Pádraig Brady stat,tail: sync with latest Linux file systems Update with the results from: kgit='https://git.kernel.org/cgit/linux/kernel/git' wget -q $kgit/torvalds/linux.git/plain/include/uapi/linux/magic.h \ -O src/fs-latest-magic.h make src/fs-magic-compare * src/stat.c (human_fstype): Add entries for: BALLOON_KVM, CGROUP2, DAXFS, ZSMALLOC. 2016-11-27 Pádraig Brady build: fix potential factor build failure on arm and risc * src/longlong.h: Sync from gmp repo incorporating: Protect umul_ppmm with do ... while (0) Replace obsolete ARC asm 'J' constraints with 'Cal' Provide umul_ppmm for riscv64 2016-11-27 Pádraig Brady shred,sort: ensure faster unaligned access to rand module glibc has changed the public define from _STRING_ARCH_unaligned to _STRING_INLINE_unaligned as per https://sourceware.org/bugzilla/show_bug.cgi?id=19462 * gl/lib/rand-isaac.c: Cater for both defines. * gl/lib/randread.c: Likewise. * src/system.h: Update commented out code. 2016-11-26 Paul Eggert shuf: test input-closed bug Problem reported by Alex Ryan (Bug#25029). * tests/misc/shuf.sh: Test for shuffling with stdin closed. build: update gnulib submodule to latest numfmt: pacify Sun C 5.14 * src/numfmt.c (main): Don't implicitly coerce pointer in bool initializer. Although it's portable C99 code, it's confusing. 2016-11-25 Pádraig Brady tests: fix false failure on new ls test * tests/ls/quote-align.sh: Remove "total" line which can vary per file system depending on allocation. Reported by Assaf Gordon on OpenSolaris (5.11/5.10). 2016-11-25 Pádraig Brady pr: fix read from invalid memory with tabs in separator This was detected with: echo a > a; pr "-S$(printf "\t\t\t")" a -m a > /dev/null Resulting in ASAN triggering: ==================================================== ERROR: AddressSanitizer: global-buffer-overflow READ of size 1 at 0x00000041b622 thread T0 #0 0x40506a in print_sep_string ../src/pr.c:2241 #1 0x407ec4 in read_line ../src/pr.c:2493 #2 0x40985c in print_page ../src/pr.c:1802 #3 0x40985c in print_files ../src/pr.c:1618 #4 0x4036e0 in main ../src/pr.c:1136 * src/pr.c (init_parameters): Ensure we only override the specified separator when it's a single tab, thus matching the calculated separator length. * tests/pr/pr-tests.pl: Add a test case. * NEWS: Mention the fix. 2016-11-24 Pádraig Brady ptx: fix an invalid heap reference with short --width * src/ptx.c (fix_output_parameters): Ensure line_width doesn't go negative, which can happen when the --width is less than the --gap-size. * tests/misc/ptx-overrun.sh: Add a test case that triggers with ASAN. (Note the longer filename is needed to trigger). Fixes http://bugs.gnu.org/25011 2016-11-24 Bernhard Voelker doc: clarify that readlink's --quiet option is on by default * src/readlink.c (usage): Mark the --quiet/--silent option as active by default. * doc/coreutils.texi (readlink invocation): Likewise. 2016-11-24 Pádraig Brady tests: fix false failure with ASAN in rm-readdir-fail * tests/rm/rm-readdir-fail.sh: ASAN correctly indicated that fts was writing to freed memory. This was because we reused a single dirent in our readdir() test wrapper. Since fts was deallocating those dirents, we now get a new dirent for each call to our readdir wrapper. 2016-11-24 Pádraig Brady split: fix memory corruption during chunk extraction ASAN reported this error for: split -n2/3 /dev/null ERROR: AddressSanitizer: negative-size-param: (size=-1) #0 0x7f0d4c36951d in __asan_memmove (/lib64/libasan.so.2+0x8d51d) #1 0x404e06 in memmove /usr/include/bits/string3.h:59 #2 0x404e06 in bytes_chunk_extract src/split.c:988 #3 0x404e06 in main src/split.c:1626 Specifically there would be invalid memory access and subsequent processing if the chunk to be extracted was beyond the initial amount read from file (which is currently capped at 128KiB). This issue is not in a released version, only being introduced in commit v8.25-4-g62e7af0 * src/split.c (bytes_chunk_extract): The initial_read != SIZE_MAX should have been combined with && rather than ||, but also this condition is always true in this function so remove entirely. * tests/split/b-chunk.sh: Add a test case. Fixes http://bugs.gnu.org/25003 2016-11-24 Pádraig Brady tail: fix checking of remoteness when not using inotify In recent commit v8.25-93-g7fc7206 we used the f->remote flag which wasn't set in all cases. This was detected with ASAN giving this error when reading f->remote; runtime error: load of value 190, which is not a valid value for type '_Bool' * src/tail.c (fremote): Query the system even without inotify. (recheck): Always set f->fremote for valid files. 2016-11-23 Jim Meyering build: avoid racy failure of "make -jN install" Installing with -j2 or greater could result in a failure like /bin/sh: line 29: /P/bin/install: Permission denied when /P/bin/install specifies your PATH-selected install program. This would arise because we're using "install" to install all man/*.1 files, and that command would run concurrently with the one that installs "/P/bin/install" itself. We would run this command: "src/ginstall src/ginstall /P/bin/install", and it would result in intervals during which the destination file does not exist, is empty or incomplete and not executable. We addressed this problem long ago for installation of actual binaries by telling the installation rules to use our just-built bin/ginstall (only when not cross-compiling) rather than the PATH-resolved "install" program. This change is to do the same for those .1 files. * src/local.mk (INSTALL): Override automake's default of something like "INSTALL = /P/bin/install -c". (INSTALL_PROGRAM): Now that we set INSTALL, there is no longer any need to set this derived variable. Its default definition, "INSTALL_PROGRAM = ${INSTALL}" does what we require. Improved by Eric Blake. 2016-11-23 Bernhard Voelker maint: fix 'syntax-check' targets for VPATH builds * cfg.mk (sc_gitignore_missing): Add $(srcdir) to .gitignore filename. (sc_gitignore_redundant): Likewise. maint: add build-aux/ar-lib to .gitignore * .gitignore: Add entry for the above file which is created by './bootstrap'. 2016-11-23 Paul Eggert pr: fix integer overflow in buffer size calcs Problem reported by Marcel Böhme (Bug#24996). * configure.ac (WERROR_CFLAGS): Avoid -Wtype-limits. * src/pr.c (col_sep_string): Now a const pointer. All uses changed. (integer_overflow): New function. (separator_string, main, init_parameters, init_store_cols): Check for integer overflow. (align_column, read_line, print_stored): Avoid integer overflow. 2016-11-22 Bernhard Voelker comm: add --total option * src/comm.c (total_option): Add bool variable for the new option. (TOTAL_OPTION): Add enum value. (long_options): Add array element for the new option. (usage): Document the new option here. (compare_files): Count the lines in total[3], and output the summary at the end. (main): Accept the new option. * doc/coreutils.texi (comm invocation): Document it. * tests/misc/comm.pl: Test it. While at it, improve the test data to have 1 unique line in the first file, 2 unique lines in the second file, and 3 common lines. * NEWS (New Features): Mention the new option. Fixes http://bugs.gnu.org/24929 2016-11-22 Pádraig Brady all: update gnulib submodule to latest Also sync these copies with gnulib: * bootstrap: Sync missed part of AIX support. * tests/init.sh: Support compare_ on AIX. tests: fix false failure with FreeBSD 11 multibyte quoting * tests/misc/printf-quote.sh: FreeBSD 11 was seen to treat \u0378 as a printable character. Therefore change to using the \u0081 C1 control character. We use the UTF-8 representation because our printf implementation explicitly disallows \u0081 as input. Reported by Assaf Gordon 2016-11-22 Pádraig Brady tests: fix false fails due to passing env vars to returns_ On BSD /bin/sh it was seen that unexported env vars passed to returns_() would not be propagated to the wrapped command. * cfg.mk (sc_prohibit_env_returns): Add a syntax check to disallow. * tests/misc/csplit-io-err.sh: Rearrange to export vars in a subshell. * tests/rm/rm-readdir-fail.sh: Likewise. * tests/misc/nohup.sh: Export and unset vars around returns_. * tests/misc/printenv.sh: Likewise. Reported by Assaf Gordon 2016-11-22 Pádraig Brady tests: fix recent tail-2 test regressions * tests/tail-2/pipe-f.sh: Avoid issue with readable directories on BSD systems. * tests/tail-2/retry.sh: Likewise. Reported by Assaf Gordon 2016-11-22 Pádraig Brady ls: improve alignment of quoted names This provides better alignment when some names are quoted, which also provides better indication that quotes are not part of the name. * src/ls.c (align_variable_outer_quotes): A new variable set when ls is aligning columns (not using -m, non-zero -w), and has a variable quoting style (shell, shell-escape, c-maybe). (quote_name_buf): Writes to buffer rather than FILE, taking care to avoid data copying if possible. Refactored from... (quote_name): ...here. This now manages the buffer passed to quote_name_buf() and outputs the padding, colors and name in the appropriate order, while managing the --dired offsets. (get_color_indicator): A new function to return the color sequence, refactored from... (print_color_indicator): ...here. This now simply outputs. (print_dir): Refactor common parts to quote_name(). (clear_files): Reset the flag indicating at least one file is quoted in the current directory. (needs_quoting): A new function to indicate at the scan stage whether a name needs quoting. Called from... (gobble_file): ...here, until we find the first quoted file. (print_name_with_quoting): Mostly refactored to quote_name(). * tests/ls/quote-align.sh: A new test for various output formats. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. 2016-11-22 Pádraig Brady b2sum: a new checksum utility with md5sum like interface Note we don't support the --algorithm option of the b2sum command in the external BLAKE2 project, as that was deemed too confusing for users. "BLAKE2b" was chosen as the default algorithm to use, which is single threaded but performs well on 64 bit. * src/blake2: CC0 source copied from external project. * cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/ from syntax checks, make update-copyright, etc. * src/local.mk: Reference the sources for b2sum, and set the compilation flags. * doc/coreutils.texi (b2sum invocation): Reference the md5sum invocation node, and add descriptions of -l. * tests/misc/b2sum.sh: Add new test. * tests/local.mk: Reference new test. * AUTHORS: Add new binary. * README: Likewise. * build-aux/gen-lists-of-programs.sh: Likewise. * man/.gitignore: Likewise. * scripts/git-hooks/commit-msg: Likewise. * man/b2sum.x: New man page template. * man/local.mk: Reference new template. * src/.gitignore: Ignore new binaries. * src/blake2/.gitignore: Ignore new build atrifacts. * src/md5sum.c (usage): Describe the new -l option. * NEWS: Mention the new program. 2016-11-22 Jim Meyering maint: avoid "make distcheck" failure due to excess .deps directories * Makefile.am (my-distcheck): Remove all .deps directories before the recursive diff that searches for left-behind files. Otherwise, with automake master (some time after v1.15), "make distcheck" would fail due to those directories being left behind after "make distclean". 2016-11-19 Pádraig Brady cp: improve status message when omitting directories * src/copy.h (cp_options): Add a new flag for install(1). * src/copy.c (copy_internal): For cp, also output remediation advice which also indicates why directories aren't copied by default. The message is unchanged for install(1). * src/cp.c (cp_option_init): Init install_mode to false. * src/mv.c (cp_option_init): Likewise. * src/install.c (cp_option_init): Init install_mode to true. * tests/install/basic-1.sh: Add a test case. * tests/cp/link-deref.sh: Adjust test case. Fixes http://bugs.gnu.org/24958 2016-11-16 Pádraig Brady tests: fix non portable date in recent test * tests/misc/date.pl: Years < 1900 fail on solaris at least. Reported by Assaf Gordon 2016-11-15 Pádraig Brady tests: fix false failure on OS X * tests/misc/sort-h-thousands-sep.sh: The sv_SE locale may be available while sv_SE.utf8 is not. Therefore test and use the same locale, now definfed in a variable. Reported by Jim Meyering. 2016-11-15 Antonio Ospite dircolors: highlight Motion JPEG multimedia files * src/dircolors.hin: Add .mjpg and .mjpeg multimedia files. This resolves http://bugs.gnu.org/24436. 2016-11-14 Jim Meyering tests: use "returns_" rather than explicit comparison with "$?" The previous "returns_"-using change failed to convert many uses of "$?". Convert all but two of the remaining ones. * tests/ls/stat-vs-dirent.sh: Likewise. * tests/misc/head-write-error.sh: Likewise. * tests/misc/nice.sh: Likewise. * tests/misc/nohup.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/misc/sync.sh: Likewise. * tests/tail-2/pid.sh: Likewise. * tests/tail-2/wait.sh: Likewise. Thanks to Bernhard Volker for spotting this. 2016-11-14 Pádraig Brady all: reduce usage of proper_name_utf8() This reduces a standard coreutils install size by about 160K. * src/cat.c: Change to proper_name() which removes about 18K text. * src/cp.c: Likewise. * src/df.c: Likewise. * src/du.c: Likewise. * src/getlimits.c: Likewise. * src/realpath.c: Likewise. * src/split.c: Likewise. * src/stdbuf.c: Likewise. * src/timeout.c: Likewise. * src/truncate.c: Likewise. * src/local.mk: Remove -llibiconv from the above programs. * cfg.mk (sc_check-AUTHORS): Adjust to use factor(1). * AUTHORS: Adjust to use ASCII to satisfy sc_check-AUTHORS. 2016-11-13 Bernhard Voelker maint: add 'returns_' to exclude list in sc_prohibit_and_fail_1 The previous commit v8.25-96-g22063c8 lets 'make syntax-check' fail, because the above check falsely detects a case where 'returns_' is already in use. * cfg.mk (sc_prohibit_and_fail_1): Add 'returns_' to exclude list. 2016-11-13 Jim Meyering tests: use "returns_" rather than explicit comparison with "$?" * tests/chmod/no-x.sh: Likewise. * tests/cp/no-deref-link1.sh: Likewise. * tests/cp/no-deref-link2.sh: Likewise. * tests/cp/no-deref-link3.sh: Likewise. * tests/du/move-dir-while-traversing.sh: Likewise. * tests/ls/infloop.sh: Likewise. * tests/ls/stat-failed.sh: Likewise. * tests/ls/time-style-diag.sh: Likewise. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/env-null.sh: Likewise. * tests/misc/env.sh: Likewise. * tests/misc/nice-fail.sh: Likewise. * tests/misc/printenv.sh: Likewise. * tests/misc/timeout.sh: Likewise. * tests/tail-2/follow-name.sh: Likewise. 2016-11-10 Pádraig Brady tail: only retry file open if --retry specifed * src/tail.c (tail_file): On failure to open a file, set ignore=true when --retry is not specified. * tests/tail-2/assert-2.sh: Adjust to the new behavior. * tests/tail-2/retry.sh: Add a test case. Also change from `tail ... && fail=1` to the more robust `returns_ 1 ...` construct which detects segfaults etc. * NEWS: Document the fix. 2016-11-09 Pádraig Brady tail: ensure -f --retry 'missing' handles truncation * src/tail.c (tail_forever): The BLOCKING optimization is only enabled for non regular files (which can't be truncated), so ensure we don't enable that unless we've a valid file descriptor. * tests/tail-2/retry.sh: Add a test case. * NEWS: Mention the bug fix. tail: avoid outputting repeated data with remote files * src/tail.c (tail_forever): Only read up to st_size on network file systems to avoid the issue with a stale attribute cache returning a smaller st_size than we have already read(). The was seen with glusterfs at least and caused the complete file to be repeatedly output due to assuming the file was truncated in this case. * NEWS: Mention the fix. 2016-11-08 Pádraig Brady tail: terminate when following pipes and untailable non pipes * src/tail.c (ignore_pipe_or_fifo): Mark the descriptor as -1 for pipes so that any_live_files() detects correctly that the entry is no longer live. * tests/tail-2/pipe-f.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/24903 which was detected using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University. 2016-11-07 Pádraig Brady maint: misc tweaks to recent changes * doc/coreutils.texi(cut invocation): Give a more accurate description of cut field handling limitations. * tests/misc/stat-fmt.sh: Fix the test header to be more general. * tests/tail-2/retry.sh: Spellings. maint: refactor printing of backup suffix --help * src/system.h (emit_backup_suffix_note): A new function to output the backup suffix info. The strings are unchanged, so translations are not impacted. * src/cp.c (usage): Use the new function. * src/ln.c (usage): Likewise. * src/mv.c (usage): Likewise. * src/install.c (usage): Likewise. 2016-11-07 Rishabh Dave maint: simplify handling of backup --suffix in various tools * src/cp.c (main): Avoid the getenv("SIMPLE_BACKUP_SUFFIX") call, which is now done if needed in the gnulib backupfile module. Also avoid the redundant strdup, as we don't modify this suffix. * src/install.c (main): Likewise. * src/ln.c (main): Likewise. * src/mv.c (main): Likewise. Fixes http://bugs.gnu.org/23153 2016-11-07 Pádraig Brady date: add %q to output the quarter of the year * doc/coreutils.texi (date invocation): Document %q. * src/date.c (usage): Likewise. * tests/misc/date.pl: Add a test case. * cfg.mk (sc_strftime_check): Adjust to allow %q. * NEWS: Mention the new feature. 2016-11-07 Pádraig Brady all: update gnulib submodule to latest Incorporating these coreutils specific changes: backupfile: initialize default suffix within the implementation strftime,strptime: support %q to represent the quarter Also sync these copies with gnulib: * bootstrap: Now supports AIX. * gnulib: Update to the latest. * tests/init.sh: dash(1) is now disabled. 2016-11-05 Pádraig Brady doc: clarify in texinfo that cut doesn't reorder fields * doc/coreutils.texi (cut invocation): Clarify the more sophisticated field processing supported by awk(1). Fixes http://bugs.gnu.org/24881 stat: make --format=%N honor the QUOTING_STYLE env var * doc/coreutils.texi (stat invocation): Describe the QUOTING_STYLE values now supported. * src/stat.c (getenv_quoting_style): A new function called from main, that sets the default quoting style for quotearg. (main): Call getenv_quoting_style() when %N specified. * tests/misc/stat-fmt.sh: Add a test case. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/23422 doc: improve the --help description of ls -N * src/ls.c (usage): The existing description was inaccurate as -N does not directly impact whether control characters are displayed or not. 2016-11-04 Pádraig Brady dd: warn about counts specified with confusing 0x prefix * src/dd.c (parse_integer): Suggest to use "00x" instead of "0x", which is significant for the "count", "seek", and "skip" operands. * tests/dd/misc.sh: Add a test case. Fixes http://bugs.gnu.org/24874 2016-10-26 Pádraig Brady md5sum,sha*sum: fix --ignore-missing with checksums starting with 00 * NEWS: Mention the fix. * src/md5sum.c (digest_file): Add a new MISSING parameter to return whether the file was missing, separately from the digest. * tests/misc/md5sum.pl: Add a test case. Fixes http://bugs.gnu.org/24795 2016-10-22 Bishop Bettini doc: fix typo in stdbuf man page * man/stdbuf.x: Correct spelling of immediately. Fixes http://bugs.gnu.org/24763 2016-10-16 Jim Meyering maint: we use die; remove now-unnecessary uses of abort * src/expr.c (integer_overflow): Remove an abort-after-die. * src/paste.c (write_error): Likewise. * src/sort.c (badfieldspec): Likewise. * src/tr.c (string2_extend): Likewise. Also remove a few break statements, each after an abort. 2016-10-16 Pádraig Brady all: use die() rather than error(EXIT_FAILURE) die() has the advantage of being apparent to the compiler that it doesn't return, which will avoid warnings in some cases, and possibly generate better code. * cfg.mk (sc_die_EXIT_FAILURE): A new syntax check rule to catch any new uses of error (CONSTANT, ...); 2016-10-16 Jim Meyering build: also distribute new file, src/die.h * src/local.mk (noinst_HEADERS): Add die.h. 2016-10-15 Jim Meyering build: add die.h; avoid new warnings from GCC 7 * src/die.h (die): New file/function from grep. Note: we expect this file to migrate to gnulib. * src/csplit.c: Include die.h. (check_format_conv_type): Use die in place of error-nonzero;break; * src/install.c (strip): Likewise. * src/nl.c (proc_text): Likewise. This also suppresses a new warning from GCC 7's -Werror=strict-overflow. * src/tail.c (parse_options): Likewise. * src/basename.c (main): Adjust "fall through" comment so that GCC 7's -Wimplicit-fallthrough honors it. * src/cp.c (main): Add a "fall through" comment. * src/ls.c (gobble_file): Likewise. (get_funky_string): Adjust a "fall through" comment so it is recognized. * cfg.mk (exclude_file_name_regexp--sc_system_h_headers): Add die.h to this list of exempt src/*.h files. maint: pr: avoid new GCC 7 warning * src/pr.c (main): Avoid this warning from GCC 7: src/pr.c:1119:6: error: assuming signed overflow does not occur when \ simplifying conditional to constant [-Werror=strict-overflow] if (n_files == 0) gnulib: update to latest and adapt printf-quote.sh test * .gitignore: Add /lib/limits.h. * tests/misc/printf-quote.sh: Reflect that a single quote is now rendered as "'", rather than as ''\'''. 2016-10-04 Pádraig Brady rm: disallow --n alias for --no-preserve-root * src/rm.c (main): Ensure the full --no-preserve-root option is specified, rather than allowing --n etc. * tests/rm/r-root.sh: Add a test case. * NEWS: Mention the change in behavior. Improved by Jim Meyering. Fixes http://bugs.gnu.org/24604 2016-09-28 Pádraig Brady tail: -F now always processes initially untailable files which was not the case when inotify was not available. * src/tail.c (any_live_files): Simplify, since the IGNORE flag is now only set when a file should be ignored indefinitely. (recheck): Only output the "giving up on name" message when that's actually the case. Only set the IGNORE flag when ignoring a file indefinitely. (tail_file): Likewise. * tests/tail-2/retry.sh: Add a test case. Also run all existing test cases with and without inotify. NEWS: Mention the fix. THANKS.in: Add the reporter. Fixes http://bugs.gnu.org/24495 which was detected using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University. 2016-09-22 Jim Meyering who: avoid new warning from upcoming gcc-7 * src/who.c (idle_string): This function would fail to compile with -Werror and today's built-from-git gcc due to this warning: src/who.c: In function 'print_user': src/who.c:201:36: error: may write format character ':' at offset 4 \ past the end of the destination [-Werror=format-length=] sprintf (idle_hhmm, "%02d:%02d", ^~~~~ The fix is to use an assertion to inform gcc of the existing invariant that guarantees the number of hours is less than 24. 2016-09-19 Paul Eggert ls: fix %%b format and precompute more The old code mishandled --time-spec='+%%b', as it misinterpreted the '%b' as being the month abbreviation. Also, it mishandled the extremely unlikely case of a month abbreviation containing '%'. The performance part of this patch sped up 'ls' by about 1% on my little benchmark of 'ls -lR' on the source directory in the en_US.UTF-8 locale (Fedora 24 x86-64). * NEWS: Document the bug fix. * src/ls.c (first_percent_b, abformat_init): New static functions. (ABFORMAT_SIZE): New constant. (use_abformat): New static var. (abmon, required_mon_width): Remove these static vars. (abmon_init): Now accepts a pointer to abmon, and returns a boolean. All callers changed. Reject month abbrs containing '%', as these would mess up strftime. Simplify mbsalign result checking, since (size_t) -1 exceeds ABFORMAT_SIZE. (abformat_init, align_nstrftime): Precompute all 24 formats at startup, rather than computing a format for each time stamp. (decode_switches): Call abformat_init instead of abmon_init. (align_nstrftime): Accept recentness bool instead of format. All callers changed. * tests/misc/time-style.sh: Test for format with '%%b'. 2016-09-11 George Burgess IV pinky,who: remove redeclarations of ttyname Given that `ttyname` is already conditionally declared in src/system.h, other declarations are redundant and problematic for example in ChromeOS which has a new FORTIFY implementation that, as an artifact of how it's implemented, causes the compiler to complain if certain standard library functions are redeclared without special compiler-specific attributes. * src/pinky.c: Remove declaration (which was unused anyway). * src/who.c: Remove declaration. * src/system.h (getlogin, getuid, geteuid, getgrgid, getpwuid, ttyname): Add the parameter type to the declarations to avoid warnings when these backup declarations are used. 2016-09-10 Jim Meyering gnulib: update to latest, for getprogname fixes Compiling in gnulib-tests/ would fail on some platforms due to a missing lib/assert.h. Reported by Assaf Gordon in https://lists.gnu.org/archive/html/coreutils/2016-09/msg00016.html 2016-09-09 Paul Eggert cp: use FICLONE instead of BTRFS_IOC_CLONE This doesn't affect the generated code on my system; it's merely a cleaner way to use the recently-introduced Linux-specific API. * m4/jm-macros.m4 (gl_CHECK_ALL_HEADERS): Check for linux/fs.h. * src/copy.c: Include if available. (FICLONE) [__linux__]: Define if not already defined. (clone_file): Use FICLONE instead of the older BTRFS_IOC_CLONE, as this ioctl is no longer btrfs-specific. 2016-09-08 Assaf Gordon date: add '--debug' option Usage example: date --debug -d 'now + 3 days' Print parsing and debugging information to stderr when using date's -d/--date option. See: http://lists.gnu.org/archive/html/coreutils/2016-01/msg00076.html * src/date.c (main): Add '--debug' option, enable debugging in gnulib's parse-datetime.y module. * tests/misc/date.pl: Repeat tests with '--debug' enable, ensure no regression. * tests/misc/date-debug.sh: Test output of '--debug' option. * tests/local.mk: Add above test. * NEWS: Mention new option. * doc/coreutils.texi: Likewise. 2016-09-08 Paul Eggert sort: port to strict C + valgrind * bootstrap.conf (gnulib_modules): Add flexmember. * src/sort.c: Include flexmember.h. (struct tempnode): Make the last member flexible. (create_temp_file): Port to strict C11/C99 rules for allocation alignment with flexible array members. build: update gnulib submodule to latest 2016-09-07 Pádraig Brady factor: reinstate immediate output with interactive use * src/factor.c (lbuf_putc): Only buffer more than a line when not using the tool interactively. * NEWS: Mention the bug fix. Fixes http://pad.lv/1620139 2016-09-07 Kamil Dudka ls: allow interruption when reading slow directories Postpone installation of signal handlers until they're needed. That is right before the first escape sequence is printed. * src/ls.c (signal_setup): A new function refactored from main() to set and restore signal handlers. (main): Move signal handler setup to put_indicator() so that the default signal handling is untouched as long as possible. Adjusted condition for restoring signal handlers to reflect the change. (put_indicator): Install signal handlers if called for the very first time. It uses the same code that was in main() prior to this commit. * NEWS: Mention the improvement. See https://bugzilla.redhat.com/1365933 Fixes http://bugs.gnu.org/24232 2016-09-01 Pádraig Brady doc: fix typo in date example * doc/coreutils.texi (date invocation): Add a missing quotation mark. Fixes http://bugs.gnu.org/24349 2016-08-31 Jim Meyering dircolors: recognize .zst and .tzst suffixes * src/dircolors.hin: Add .zst and .tzst suffixes for the Zstandard/zstd compression tools. See http://zStd.net and https://code.facebook.com/posts/1658392934479273/ 2016-08-30 Jim Meyering maint: tr: tweak/make-consistent a pair of loop indices * src/tr.c (unquote): Use indices of type "unsigned int", rather than a mix or "size_t" and "int", presuming that no command-line- specified string will have length longer than UINT_MAX. tr: move decls "down" 2016-08-29 Jim Meyering ptx: avoid new warning/error from upcoming gcc-7.x * src/ptx.c (fix_output_parameters): Switch to an unsigned type that matches the OCCURS.file_index type. This avoids the following error from gcc-7.0.0 20160829 (experimental): src/ptx.c:1220:14: error: assuming signed overflow does not occur \ when simplifying conditional to constant [-Werror=strict-overflow] if (file_index > 0) 2016-08-18 Pádraig Brady cp: with --parents --no-preserve=mode don't copy dir perms * src/cp.c (make_dir_parents_private): Use default permissions for created directories when --no-preserve=mode is specified. * tests/cp/cp-parents.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/24251 doc: fix ambiguous multiplier info in dd man page * src/dd.c (usage): Add a comma which is significant for the rewrapping done in man pages. 2016-08-09 Assaf Gordon maint: refactor common expand(1) and unexpand(1) code * src/expand.c, src/unexpand.c: Move global variables from here... * src/expand-common.h, src/expand-common.c: ... to here. * src/expand.c, src/unexpand.c: (parse_tab_stops, validate_tab_stops, next_file): Move identical functions to new module. (add_tab_stop): Move to new module, including additional code from 'unexpand' (keeping max_column_width) which will have no effect in when used in 'expand'. Refactor common next-column calculation code into a new function 'get_next_tab_column'. * src/local.mk: (src_expand_SOURCES, src_unexpand_SOURCES): Add 'expand-common.c'; (noinst_HEADERS): Add 'expand-common.h'. * po/POTFILES.in: Add 'expand-common.c'. * tests/misc/expand.pl: Add more tests. * tests/misc/unexpand.pl: Likewise. * TODO: Move conclusions to above test after investigation. 2016-08-03 Philipp Thomas df: improve performance with many mount points Use hash table for seaching in filter_mount_list() and get_dev() This improves performance for 20K mount entries from: real 0m1.731s user 0m0.532s sys 0m1.188s to: real 0m1.066s user 0m0.028s sys 0m1.032s * src/df.c (devlist_table): Define hash table. (devlist_hash): Add hash function. (devlist_compare): Add hash comparison function. (devlist_for_dev): Add lookup function. (devlist_free): Add cleanup function. (filter_mount_list): Use the above hash table. While at it, rename the variable 'devlist' to 'seen_dev' for better readability. (me_for_dev): Use the above lookup function. NEWS: Mention the improvement. THANKS.in: Remove the committer; add original submitter Josef Cejka. 2016-07-28 Pádraig Brady doc: revert recent date(1) synopsis change * src/date.c (usage): This was thought to introduce other ambiguities, and was inconsistent with the format presented in the touch(1) docs. * doc/coreutils.texi (date invocation): Likewise. See http://bugs.gnu.org/24077 2016-07-27 Bernhard Voelker maint: pacify GCC 6 with -Wnull-dereference src/id.c:249:29: error: potential null pointer dereference \ [-Werror=null-dereference] pw_name = xstrdup (pwd->pw_name); ~~~^~~~~~~~~ src/whoami.c:89:11: error: potential null pointer dereference \ [-Werror=null-dereference] puts (pw->pw_name); ~~^~~~~~~~~ * src/id.c (main): Explicitly exit with EXIT_FAILURE after an eror to help gcc-6 to detect that the dereferenced pointer is valid. * src/whoami.c (main): Likewise. 2016-07-27 Bernhard Voelker build: update gnulib submodule to latest Mainly for * 77d36b5 obstack: pacify GCC 6 with -Wnull-dereference 2016-07-27 Jim Meyering maint: skip a check when en_US.UTF-8 collation rules are broken * cfg.mk (sc_THANKS_in_sorted): This check would fail on systems for which "." is not ignored. Add a quick sort-based check for that error, and skip the check on any broken system. 2016-07-27 Pádraig Brady doc: make date(1) synopsis consistent with output +FORMAT * src/date.c (usage): Make the characters used to summarize the input format, match the output +FORMAT characters. * doc/coreutils.texi (date invocation): Likewise. Fixes http://bugs.gnu.org/24077 2016-07-27 Bernhard Voelker maint: fix dependency of man/arch.1 The following is a reproducer for the wrong dependency: $ ./configure --enable-install-program=arch $ make $ rm -f src/arch man/arch.1 $ make man/arch.1 GEN man/arch.1 help2man: can't get `--help' info from man/arch.td/arch Try `--no-discard-stderr' if option outputs to stderr Makefile:14378: recipe for target 'man/arch.1' failed make: *** [man/arch.1] Error 127 * man/local.mk (man/arch.1): Change to depend on src/arch rather than src/uname: while the arch binary depends on uname.c and uname-arch.c, its man page depends on the arch binary. Reported downstream by Rodrigues Goldwyn in https://build.opensuse.org/request/show/415172 2016-07-18 Kamil Dudka sort: with -h, disallow thousands separator between number and unit * src/sort.c (traverse_raw_number): Accept thousands separator only if it is immediately followed by a digit. * tests/misc/sort-h-thousands-sep.sh: Cover the fix for this bug. Suggested by Pádraig Brady in http://bugs.gnu.org/24015 2016-07-18 Kamil Dudka sort: make -h work with -k and blank used as thousands separator * src/sort.c (traverse_raw_number): Allow to skip only one occurrence of thousands_sep to avoid finding the unit in the next column in case thousands_sep matches as blank and is used as column delimiter. * tests/misc/sort-h-thousands-sep.sh: Add regression test for this bug. * tests/local.mk: Reference the test. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1355780 Fixes http://bugs.gnu.org/24015 maint: sort.c: deduplicate code for traversing numbers * src/sort.c (traverse_raw_number): New function for traversing numbers. (find_unit_order): Use traverse_raw_number() instead of open-coding it. (debug_key): Likewise. 2016-07-08 Kamil Dudka install: with -Z, set default SELinux context for created directories * doc/coreutils.texi (install invocation): Update -Z documentation. * src/install.c (make_ancestor): Set default security context before calling mkdir() if the -Z option is given. (process_dir): Call restorecon() on the destination directory if the -Z option is given. (usage): Update -Z documentation. * tests/install/install-Z-selinux.sh: A new test for 'install -Z -D' and 'install -Z -d' based on tests/mkdir/restorecon.sh. * tests/local.mk: Reference the test. * NEWS: Mention the improvement. Reported at https://bugzilla.redhat.com/1339135 Fixes http://bugs.gnu.org/23868 2016-07-06 Pádraig Brady tests: avoid false failure on 32 bit in readdir() test * tests/rm/rm-readdir-fail.sh: Since we use the returned dirent from the readdir wrapper it must be the correct type and not just cast. Therefore setup so that we only have to define a wrapper for readdir(), which works appropriately on 32 and 64 bit. Issue reported by Bernhard Voelker, where rm was seen to invoke rmdir() on invalid file names. 2016-07-02 Assaf Gordon tests: improve recent seq EPIPE test Follow-up to commit c92585b1: improve epipe test script, with suggestions from Bernhard Voelker and Pádraig Brady. see: http://lists.gnu.org/archive/html/coreutils/2016-04/msg00067.html . * tests/misc/seq-epipe.sh: Avoid too many subshells, stricter validation of errors and output from seq, skip if trapping SIGPIPE is not supported. 2016-06-29 Pádraig Brady stty: fix sane setting of susp to ^z on Solaris * src/stty.c: Disable setting of "swtch" to ^z as that conflicts with and disables using ^z as "susp". * NEWS: Mention the bug fix. Reported and tested by Rich Burridge at: http://bugs.gnu.org/23866 2016-06-26 Peter Benie tests: verify that fts diagnoses readdir() failures * tests/rm/rm-readdir-fail.sh: A new test to simulate readdir() failing immediately or after returning a few entries, and verifying that rm does the appropriate thing. This was initially reported at: http://bugzilla.opensuse.org/show_bug.cgi?id=984910 where it was mentioned that readdir() may fail when an NFS server has a poor readdir cookie implementation. 2016-06-26 Pádraig Brady all: update gnulib submodule and tests/init.sh to latest * gnulib: Update to latest. * NEWS: Specifically mention the fts readdir() fix and reindent to standard indentation. * tests/init.sh: Update from gnulib. 2016-06-25 Paul Eggert maint: simplify partial writes etc. in 'yes' * src/yes.c: Include full-write.h. (main): Use full_write, not write, to simplify handling of partial writes. Don't bother using stdio to output data; just use full_write with a buffer as large as needed. Reuse operand strings if possible, and if the buffer would otherwise be large. 2016-06-24 Pádraig Brady yes: handle short writes * src/yes.c (main): Loop over the write buffer to handle the case where write may write less than requested. * NEWS: Mention the bug fix. Reported by Paul Eggert. yes: fix copy and paste issue with previous commit * src/yes.c (main): Output 'y' not '-'. * tests/misc/yes.sh: Add a test for default output. 2016-06-23 Paul Eggert maint: work even if argc == INT_MAX GCC 7 warned about undefined behavior in this unlikely case. Problem reported by Jim Meyering in: http://bugs.gnu.org/23825 * src/md5sum.c (main): * src/paste.c (main): * src/yes.c (main): Avoid undefined behavior when argc == INT_MAX. 2016-06-01 Pádraig Brady doc: clarify sort --key handling of default field separators * doc/coreutils.texi (sort invocation): Mention in the summary dicussion that --key is used to specify fields. Give a summary in the --key description, of the most common use case of specifying a field, and that by default those fields include the blank separators at the start of each field in the comparisons. 2016-06-01 Assaf Gordon sort: modify 'leading spaces' debug warning scenarios Print warning regardless of locale, avoid warning if key is zero width. Problem reported by Karl Berry in http://bugs.gnu.org/23665 . * src/sort.c: (key_warnings): change conditions for 'leading spaces' warning. * tests/misc/sort-debug-warn.sh: adjust tests accordingly. 2016-06-01 Assaf Gordon tests: sort-debug-warn: add progress information lines Easier troubleshooting of individual 'sort --debug' messages. Discussed in http://bugs.gnu.org/23665 . * tests/misc/sort-debug-warn.sh: add progress number before each sort invocation. 2016-05-15 Pádraig Brady dircolors: output colors for any $TERM matching *color* * src/dircolors.hin: Reduce the list by replacing all specific mentions of "color" with the pattern "*color*". This will also cater for other entries like "konsole-256color". Fixes http://bugs.gnu.org/23542 2016-05-14 KOBAYASHI Takashi nl: reset numbering for each section * NEWS: Mention the bug fix. * src/nl.c (proc_body, proc_footer): Reset numbering here too. * doc/coreutils.texi (nl invocation): Adjust for behavior change. * tests/misc/nl.sh: Add a test case. 2016-05-12 Pádraig Brady stat,tail: add support for the SMB2 remote file system * stc/stat.c (human_fstype): Add file system ID definition, and use "smb2" as the name. * NEWS (Improvements): Mention the change. Fixes http://bugs.gnu.org/23516 2016-05-04 Jim Meyering maint: avoid new warning from gcc (GCC) 7.0.0 20160503 (experimental) * src/id.c (main): When configured with --enable-gcc-warnings and using the very latest gcc built from git, building would fail with this: src/id.c:200:8: error: assuming signed overflow does not occur when \ simplifying conditional to constant [-Werror=strict-overflow] bool default_format = (just_user + just_group + just_group_list ^~~~~~~~~~~~~~ Rewrite to use bool-appropriate operators. 2016-04-26 Pádraig Brady tests: avoid false failure when df lists no root file system * tests/df/skip-duplicates.sh: Skip the test when there is no '/' entry listed by df, which was seen in certain chroot setups. 2016-04-24 Assaf Gordon seq: detect and report I/O errors immediately Ensure I/O errors are detected (and terminate seq), preventing seq from infloop (or running for long time with a large range) upon write errors or ignored SIGPIPE. Examples: seq 1 inf > /dev/full (seq_fast) seq 1.1 0.1 inf >/dev/full (print_numbers) * src/seq.c (io_error): A new function to diagnose appropriate stdio errors and exit the program with failure status. (seq_fast, print_numbers): Explicitly check for write errors and terminate the program with diagnostic. * tests/misc/seq-io-errors.sh: Test error detection with /dev/full. * tests/misc/seq-epipe.sh: Test error detection with broken pipes. * tests/local.mk: Add new tests. * NEWS: Mention the fix. 2016-04-22 Pádraig Brady stat,tail: add support for Windows Subsystem for Linux * src/stat.c (human_fstype): Add file system ID definition, and use "wslfs" as the name. * NEWS (Improvements): Mention the change. Fixes http://bugs.gnu.org/23273 2016-04-21 Pádraig Brady yes: avoid redundant diagnostics on write error * src/yes.c (main): For large inputs only write a single diagnostic for write errors. * tests/misc/yes.sh: Test when /dev/full is available. sleep,timeout: support overflowing floating point values * src/sleep.c (main): Allow ERANGE since we allow "inf" values. * src/timeout.c (parse_duration): Likewise. * tests/misc/sleep.sh: New file. Tests for sleep(1). * tests/misc/timeout-parameters.sh: Add case for newly allowed $LDBL_MAX. Also use returns_ throughout the file. Also avoid small timeout values which might give false failures under load. tests: avoid potential overflows with `expr` * cfg.mk (sc_prohibit_expr_unsigned): expr uses intmax_t internally when GMP is not available, so flag any uses of unsigned limits. * tests/misc/timeout-parameters.sh: Remove the overflow related to UINT_MAX as it's handled by the following case. Change the following case from 99... to $TIME_T_OFLOW to be more expressive in what it's actually testing. Directly check that commands succeed, rather than inspecting $? afterwards. * tests/dd/skip-seek-past-dev.sh: Guard against large device sizes. * tests/id/setgid.sh: Protect: Guard against large group IDs. * tests/misc/cut-huge-range.sh: Tweak comment to avoid syntax check. It's tempting to simplify to just skip the test if expr fails, but SIZE_MAX > INTMAX_MAX is the common case. 2016-04-20 Pádraig Brady tests: fix recent false failure due to a race * tests/cp/parent-perm-race.sh: This new race introduced in commit v8.25-5-g632eda5 is quite hard to hit, but is due to `ls > fifo` doing write()/close()/exit() once `cp` has open() the source fifo. Then the subsequent comparison of the destination file may fail due to the file being missing or empty. Previously `ls` generated output that was independent of `cp`. Now we must wait for `cp` to finish before inspecting the destination file that it wrote. 2016-04-18 Bernhard Voelker tests: let mkdir/p-v.sh fail when compare fails * tests/mkdir/p-v.sh: Set fail=1 when compare fails: the latter only sets a non-Zero return code; evaluating that is up to the caller. 2016-04-14 Bernhard Voelker seq: do not allow NaN arguments * src/seq.c (isnan): Define macro. (scan_arg): Add check if the given argument is NaN, and exit with a proper error diagnostic in such a case. (usage): Document it. * tests/misc/seq.pl: Add tests. * doc/coreutils.texi (seq invocation): Document the change. * NEWS (Changes in behavior): Mention the change. seq: do not allow 0 as increment value * src/seq.c (main): Exit with an error diagnostic when the given step value is Zero. (usage): Document it. * doc/coreutils.texi (seq invocation): Likewise. * tests/misc/seq.pl: Add tests. * NEWS (Changes in behavior): Mention the change. Reported by Маренков Евгений in: http://bugs.gnu.org/23110 2016-04-14 Yigal Korman stat,tail: add support for M1FS * src/stat.c (human_fstype): Add file system ID definition. * NEWS (Improvements): Mention the change, moving the previously added "prl_fs" change note from "Changes in behavior" to here. Fixes http://bugs.gnu.org/23283 2016-03-31 Pádraig Brady stat: report type for "prl_fs" file systems * src/stat.c (human_fstype): Identify the parallels file system. Also tag as remote so that tail(1) doesn't use inotify, which fails to detect changes made outside a VM. Fixes http://bugs.gnu.org/23143 2016-03-17 Paul Eggert date ls pr: fix time zone abbrs on SysV platforms The problematic code computed a struct tm in one time zone, and then printed it or converted it to a string in another. To be portable the same time zone needs to be used for both operations. On GNU platforms this is not an issue, but incorrect output can be generated on System V style platforms like AIX where time zone abbreviations are available only in the 'tzname' global variable. Problem reported by Assaf Gordon in: http://bugs.gnu.org/23035 * NEWS: Document the bug. * src/date.c (show_date): * src/ls.c (long_time_expected_width, print_long_format): * src/pr.c (init_header): * src/stat.c (human_time): Use localtime_rz instead of localtime, so that the time zone information is consistent for both localtime and time-formatting functions like fprintftime and nstrftime. For 'stat' this change is mostly just a code cleanup but it also causes stat to also print nanoseconds when printing time stamps that are out of localtime range, as this is more consistent with what other programs do. For programs other than 'stat' this fixes bugs with time zone formats that use %Z. * src/du.c, src/pr.c (localtz): New static var. (main): Initialize it. * src/du.c (show_date): New time zone argument, so that localtime and fprintftime use the same time zone information. All callers changed. * tests/misc/time-style.sh: New file. * tests/local.mk (all_tests): Add it. * tests/misc/date.pl: Test alphabetic time zone abbreviations. 2016-03-15 Jim Meyering maint: remove gl/lib/reg*.c.diff; no longer needed * gl/lib/regcomp.c.diff: Remove file, now that gnulib's regcomp.c compiles regex.c with -Wno-unused-parameter. * gl/lib/regex_internal.h.diff: Likewise. * gl/lib/regex_internal.c.diff: This file induced a change to ensure that the "Idx" type was unsigned and to remove a few "VAR < 0" comparisons. These days, it is probably fine to stay in sync with gnulib/glibc's copies of these files, so remove these patches, too. * gl/lib/regexec.c.diff: Likewise. Prompted by a report by Assaf Gordon and a suggestion from Paul Eggert. 2016-03-09 Eric Blake build: update gnulib submodule to latest Mainly for: *bdb72bc6 set-permissions: fix compilation on Cygwin * bootstrap: Sync with gnulib. * gl/lib/regcomp.c.diff: Regenerate against latest gnulib. 2016-03-07 Jim Meyering tests: avoid false-failure of split/filter.sh on XFS * tests/split/filter.sh: Use OFF_T_MAX-1 rather than OFF_T_MAX as the size of a test file, to avoid false failure on an XFS file system (or any file system permitting a file of size OFF_T_MAX). Reported as http://bugs.gnu.org/22931 2016-03-04 Eric Blake test: Document that -a and -o are undesirable POSIX recommends avoiding -a and -o, for good reason. src/test.c (usage): Mention that inherent ambiguities exist with binary -a and -o. Problem reported by Martin Gebert in: http://bugs.gnu.org/22909 2016-02-23 Paul Eggert all: be less strict about usage if POSIX 2008 sort, tail, and uniq now support traditional usage like 'sort +2' and 'tail +10' on systems conforming to POSIX 1003.1-2008 and later. * NEWS: Document this. * doc/coreutils.texi (Standards conformance, tail invocation) (sort invocation, uniq invocation, touch invocation): Document new behavior, or behavior's dependence on POSIX 1003.1-2001. * src/sort.c (struct keyfield.traditional_used): Rename from obsolete_used, since implementations are now allowed to support it. All uses changed. (main): Allow traditional usage if _POSIX2_VERSION is 200809. * src/tail.c (parse_obsolete_option): Distinguish between traditional usage (which POSIX 2008 and later allows) and obsolete (which it still does not). * src/uniq.c (strict_posix2): New function. (main): Allow traditional usage if _POSIX2_VERSION is 200809. * tests/misc/tail.pl: Test for new behavior. doc: document sort -o F F danger Problem reported by Yanyan Jiang 蒋炎岩 in: http://bugs.gnu.org/22769 * doc/coreutils.texi (sort invocation): Mention possibility of 'sort -o F F' data loss during crashes. 2016-02-16 Daniel Lockyer date: free timezone variable to avoid valgrind warning * src/date.c (main): Free TZ variable to pacify valgrind, guarded by IF_LINT. 2016-02-16 Pádraig Brady split: adjust recent --number changes * src/split.c (lines_rr): Reinstate the conditional setting of the WROTE boolean, as otherwise split -n r/1 would consume all input when all --filter commands are stopped. There was a test in place to check for this, but it was incorrect as detailed below. (input_file_size): Immediately disallow --number with non seekable inputs, as such an invocation is not currently generally supported and will fail as the data overflows the internal buffer. * tests/split/l-chunk.sh: Adjust to again disallow -n /dev/zero. Also change all '&& fail=1' checks to use the 'returns_ 1' form. * tests/split/filter.sh: Change the no longer supported /dev/zero case to a regular $OFF_T_MAX file (supported on XFS for example). Also fix the timeout(1) commands so they're not subject to pipefail issues. 2016-02-15 Göran Uddeborg stty: fix translated --help so the gettext header is not printed * src/stty.c (usage): Remove an erroneous call to translate an empty string, added in commit v8.23-112-g564f84a, which results in the gettext header being printed for translated languages. * THANKS.in: Remove the now committer. * NEWS: Mention the bug fix. 2016-02-12 Paul Eggert tests: don't wait forever on GNU/Hurd * tests/cp/parent-perm-race.sh: Add timeouts so that the test does not wait forever on GNU/Hurd. This does not fix the underlying bug but at least lets the tests make progress. Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624 Also, check that 'cp' actually copies the data; this is a better test anyway, and simplifies the test code. Also, avoid unlikely race if 'pid' is set in the environment. split: fix problems with /dev/zero Problem reported by Nelson H.F. Beebe in: http://bugs.gnu.org/22624 Other problems also fixed: basically, the code got confused because GNU/Linux reports that /dev/zero has size zero. * src/split.c (input_file_size): Now takes struct stat *, not just size. Always store the first buffer. All callers changed. Treat /dev/zero as an infinitely-large file, both on GNU/Linux where fstat and lseek say its size is zero, and on GNU/Hurd where they say the size is OFF_T_MAX. (cwrite): Return true on success. (bytes_split): Don't try to read past EOF, and stop if a write fails. (lines_rr): Omit stray check for ignorable errno. (main): Get file size only when n_units > 1, since that's the only time it is needed. Defer most of the work to input_file_size. * tests/split/l-chunk.sh: Adjust tests to match new behavior on oddball inputs. 2016-01-30 Vincent Lefevre doc: fix the --iso-8601 example in date --help * src/date.c (usage): The colon is used in the timezone offset since commit v8.24-64-g17bbf6c. * THANKS.in: Remove committer. Fixes http://bugs.gnu.org/22491 2016-01-25 Pádraig Brady build: avoid dynamic linking issue on Solaris sparc Solaris Studio 12 on sparc (not x86) will not remove unused functions, thus leaving a reference to an undefined program_name symbol from emit_try_help(). * src/system.h (emit_try_help): Change from an inline function to a macro, so that the inline function is not actually defined in libstdbuf.c. Fixes http://bugs.gnu.org/22430 2016-01-20 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.25 * NEWS: Record release date. 2016-01-19 Pádraig Brady tests: avoid false failure in tail inotify test * tests/tail-2/wait.sh: Restrict inotify specific test portion to local file systems and also not with ---disable-inotify specified. Failure noticed on NFS. tests: avoid false failure due to remove() ignoring u-w on NFS * tests/rm/rm1.sh: Also remove the group write bit which was required on one NFS setup at least. Note u-w was enough to deny file creation, g-w was also required to deny file removal. * tests/rm/cycle.sh: Likewise. * tests/mv/perm-1.sh: Likewise. tests: avoid false failure in preserve-slink-time.sh on GPFS * tests/cp/preserve-slink-time.sh: Add a delay between the ln and the cp so that there is enough difference between the timestamps so GPFS won't discard the update. Reported by Assaf Gordon. tests: avoid false failure in shred-passes.sh * tests/misc/shred-passes.sh: Specify an exact amount to shred, to avoid running out of simulated random data on file systems with a large st_blksize like GPFS for example. Reported by Assaf Gordon. 2016-01-19 Pádraig Brady tail: disable inotify with only non existent files tests/tail-2/F-headers.sh and test/tail-2/retry.sh fail on on remote file systems due to tail going into inotify mode due to not being able to determine the remoteness of the non existent files. * src/tail.c (any_non_remote_file): A new function used to disable inotify when there are no open files, as we can't determine remoteness in that case. * NEWS: Mention the bug fix. 2016-01-18 Pádraig Brady all: don't unconditionally use leaf optimization for NFS * gnulib: Update to latest where the only change is to not unconditionally enable leaf optimization for fts on NFS, as it was seen to abort() with some NFS servers as per: https://bugzilla.redhat.com/1299169 This affects utilities that traverse directories like cp, rm, chmod etc. * NEWS: Adjust the improvement message to leave only XFS. 2016-01-18 Assaf Gordon tests: adjust memory limits in head-c.sh * tests/misc/head-c.sh: Further increase the limit to 8MB over the base test command. 5.1MB was seen as minimum extra needed on FreeBSD 9.3 x86_64. 2016-01-18 Pádraig Brady tests: adjust delays to avoid false positives * tests/rm/dangling-symlink.sh: The effective delay was only 1.5s. So delay before the operation, and increase the iteration count by 1 to increase the delay to 6.3s. This failure was noticed once on a FreeBSD 10.2 x86_64 virtual machine. * tests/dd/stats.sh: Remove quotes on sleep argument to ensure arguments are accumulated appropriately. * tests/du/move-dir-while-traversing.sh: sleep before operation, to increase the effective delay from 1.6s to 3.2s. * tests/tail-2/flush-initial.sh: Likewise. * tests/tail-2/pipe-f2.sh: Sleep first so that the effective delay matches the commented value. test: skip test rather than fail with Solaris 10 sed * tests/rm/r-root.sh: This test was seen to ERROR due to sed input not having a terminating '\n'. Therefore just skip with diagnostics upon failure to adjust the error output with sed. Reported by Assaf Gordon. 2016-01-16 Pádraig Brady tests: fix false failure with spaces in absolute $PWD * tests/install/create-leading.sh: Ensure $PWD is quoted. Reported by `make dist-check`. 2016-01-16 Pádraig Brady build: update gnulib submodule to latest Mainly for: * dce117a sig2str: list all signals on FreeBSD >= 7 * cff4b38 acl-permissions: port to USE_ACL==0 platforms 2016-01-16 Pádraig Brady tests: show mount list on failure for df tests * init.cfg (dump_mount_list_): A new function to output the system mount list. * tests/df/df-symlink.sh: Call dump_mount_list_ upon failure. * tests/df/over-mount-device.sh: Likewise. * tests/df/problematic-chars.sh: Likewise. * tests/df/skip-rootfs.sh: Likewise. * tests/df/total-verify.sh: Likewise. * tests/df/unreadable.sh: Likewise. tests: avoid false failure with BTRFS subvolumes * tests/df/df-symlink.sh: Only check the symmetry of the source <-> target lookup, when the source is only mounted once, which isn't the case if '.' is a BTRFS subvolume for example. Reported by Assaf Gordon on a default OpenSUSE 42.1 install. mv: consistently warn about multiply specified source dirs * src/copy.c (copy internal): Remember directories irrespective of their link count, because on some file systems like BTRFS, directories always have a link count of 1. 2016-01-15 Pádraig Brady tests: remove a non portable localtime test * tests/misc/date.pl: This test was dependent on perl unconditionally calling the system localtime() as date(1) does. However perl has its own configure checks in place to avoid buggy localtime implementations. Therefore comment out this test as a FIXME for now. This test was seen to fail on FreeBSD 10 where localtime() returns nonsense values instead of failing for arguments >= 2^56. tests: adjust memory limits in head-c.sh * tests/misc/head-c.sh: Further increase the limit, to account for the fact that head(1) needs at least 1.576MB over the base test command on FreeBSD 10. 4MB should be enough headroom while still providing over allocation protection. Reported by Assaf Gordon. tests: simplify invalid signal determination for kill -l * src/operand2sig.c (operand2sig): Add a detailed comment explaining why we validate even very large shell exit status values. * tests/misc/kill.sh: Add a test case for the ksh scheme. Simplify the INVALID signal number determination which also avoids a false failure on systems like FreeBSD 10 with incomplete signal list (caused by inaccurate NSIG). tests: support older fallocate number formats * tests/cp/fiemap-extents.sh: Support RHEL6 fallocate which doesn't support IEC suffixes like "MiB" on numbers. Also add some extra framework_failure_ protections. Reported by Assaf Gordon. tests: avoid coredumps when determining memory limits * init.cfg (get_min_ulimit_v_): Refactor ulimit call to... (ulimit_supported_): ...here, and add calls to avoid coredumps. 2016-01-14 Pádraig Brady sort: with --debug, flag setlocale() failures on OpenBSD Locale categories are not equivalent on OpenBSD, and LC_COLLATE only supports "C" for example. Now LC_ALL is supported to set multiple other categories on OpenBSD, so setlocale(LC_ALL, "") returns a string indicating which categories were updated and which ignored. Therefore... * src/sort.c (main): ...Call setlocale(LC_COLLATE, "") to explicitly check whether a specified LC_ALL or LC_COLLATE environment variable value is supported for the LC_COLLATE category. Also use !! to explicitly convert to bool to support c89 systems where bool is an int, and thus would get values > 1. Reported by Assaf Gordon. 2016-01-13 Pádraig Brady maint: remove temporary doc/constants.texi-t during build * doc/local.mk (constants.texi): Remove temp file. maint: fix old spelling mistakes in NEWS * NEWS: s/descendent/descendant/; s/indiscriminite/indiscriminate/. * cfg.mk (old_NEWS_hash): Update via `make update-NEWS-hash`. doc: suggest dd "sync" flag to maximize "nocache" effectiveness doc/coreutils.texi (dd invocation): Add oflag=sync to the streaming example. Also reference the "direct" flag. Mention this is only a request to the system. * src/dd.c (usage): Mention the "sync" flag along with "nocache". Also mention that it's only a request to drop the cache. * THANKS.in: Add reporter Francois Rigault. doc: mention the '0#' printf flags with stat %a * src/stat.c (usage): Mention the '#' and '0' flags are useful with %a. * doc/coreutils.texi (stat invocation): Likewise. Also give an example printing unambiguous octal output. Reported at http://bugs.debian.org/810539 2016-01-13 Pádraig Brady mv: fix data loss with repeated source dir and same destination commit v8.23-31-g90aa291 failed to consider this case, where the previous rename has failed, thus causing the following to remove the specified directory: mv dir dir dir * src/copy.c (copy_internal): Assume this rename attempt has succeeded, as a previous failure will already have been handled, and we don't want to remove the source directory in this case. * tests/cp/duplicate-sources.sh: Consolidate this test file to... * tests/mv/dup-source.sh: ...here. Add test cases for same source and dest. * tests/local.mk: Remove the consolidated test. * NEWS: Mention the bug fix. Reported at https://bugzilla.redhat.com/1297464 2016-01-13 Pádraig Brady tests: include part of wc-files0.sh skipped by mistake * tests/misc/wc-files0.sh: $fail is always set to 0, so correct the test to include that part of the test case. wc: avoid ambiguous output with '\n' in file names * src/wc.c (write_counts): Shell escape the file name if it contains '\n' so only a single line per file is output. * tests/misc/wc-files0.sh: Add a test case. * NEWS: Mention the improvement. 2016-01-13 Assaf Gordon numfmt: add the -z,--zero-terminated option * doc/coreutils.texi (numfmt invocation): Reference the description. * src/numfmt.c: Parameterize '\n' references. * tests/misc/numfmt.pl: Add tests for character and field processing. * NEWS: Mention the new feature. 2016-01-13 Pádraig Brady paste: add the -z,--zero-terminated option * doc/coreutils.texi (paste invocation): Reference -z description. * src/paste.c (main): Parameterize the use of '\n'. * tests/misc/paste.pl: Add test cases. * NEWS: Mention the new feature. comm: support NUL --output-delimiter for consistency * src/comm.c (main): Track the output delimiter length, so that it can be adjusted to 1 for the NUL delimiter. Also rename the global variable from "delimiter" to "col_sep" so its use is more obvious, and to distinguish from the recently added "delim" global variable. * tests/misc/comm.pl: Adjust accordingly. comm: add the -z,--zero-terminated option * doc/coreutils.texi (comm invocation): Reference option description. * src/comm.c (main): Use readlinebuffer_delim() to support a parameterized delimiter. * tests/misc/comm.pl: Add test cases. * NEWS: Mention the new feature. tac: support an empty (NUL) --separator * doc/coreutils.texi (tac invocation): Mention the NUL delineation with an empty --separator. * src/tac.c (main): Allow an empty separator when -r not specified. * tests/misc/tac.pl: Add test cases. * NEWS: Mention the new feature. Fixes http://bugs.gnu.org/8103 cut: add the -z,--zero-terminated option * doc/coreutils.texi (cut invocation): Reference the description. * src/cut.c: Parameterize '\n' references. * tests/misc/cut.pl: Add tests for character and field processing. * NEWS: Mention the new feature. 2016-01-13 Richard Russon head,tail: add the -z,--zero-terminated option * doc/coreutils.texi: Reference the option description. * src/head.c: Parameterize the delimiter character. * src/tail.c: Likewise. * tests/misc/head.pl: Add test case. * tests/misc/tail.pl: Likewise. * NEWS: Mention the new feature. 2016-01-13 Pádraig Brady join,sort,uniq: with -z, treat '\n' as a field separator * NEWS: Mention the change in behavior. * doc/coreutils.texi (newlineFieldSeparator): A new description, referenced from ({join,sort,uniq} invocation). * src/system.h (field_sep): A new inline function to determine if a character is a field separator. * src/join.c (usage): s/whitespace/blank/ to be more accurate wrt which characters are field separators. (xfields): s/isblank/field_sep/. * src/sort.c (inittables): Likewise. * src/uniq.c (find_field): Likewise. * tests/misc/join.pl: Adjust -z test, and add a test/example for processing the whole record with field processing. * tests/misc/sort.pl: Add -z test cases, including case with '\n'. * tests/misc/uniq.pl: Add -z -f test case with \n. 2016-01-10 Bernhard Voelker tests: fix races wrt output files of background processes At least the false positive in tail-2/follow-stdin.sh could be seen on a 4-core i5 system with -j8. Fix similar cases. too. * tests/tail-2/follow-stdin.sh: Empty the 'out' file in each iteration. Otherwise, under heavy system load, 'check_tail_output' would see the expected output from the previous round before tail would have the chance to come up in the background. While at it, move the creation of the 'exp' file out of the loop. * tests/dd/stats.sh: Empty the output file of the background process here, too. * tests/misc/cat-buf.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/tail-2/wait.sh: Likewise. 2016-01-06 Pádraig Brady stty: support "flusho" local setting This is used to discard terminal output on FreeBSD based systems. * src/stty.c (mode_info): Add FLUSHO where supported. (usage): Document for "local" and "combination" settings. * doc/coreutils.texi (local stty settings): Document, mentioning that this option is supported but ignored on GNU/Linux. 2016-01-06 Pádraig Brady stty: support [-]drain setting to control waiting for pending Tx Instead of commit v8.24-132-g5171bef which only provides control to disable this behavior (with -I), provide the symmetrical "[-]drain" special setting. * src/stty.c (main): Parse the [-]drain setting instead of -I, and treat like a global option. (usage): Adjust accordingly. * tests/misc/stty.sh: Test "drain" with and without options. * NEWS: Mention the new feature. 2016-01-04 Pádraig Brady stty: support -I, --immediate to not wait for pending transmission In some cases an initial drain may block indefinitely as discussed at: http://lists.gnu.org/archive/html/coreutils/2016-01/msg00007.html * src/stty.c (main): Use TCSANOW rather than TCSADRAIN if -I specified. (usage): Document the new option. * doc/coreutils.texi (stty invocation): Likewise. * tests/misc/stty.sh: Ensure -I is supported. * NEWS: Mention the new feature. 2016-01-03 Pádraig Brady doc: avoid unwanted display of the 'coreutils' command info * doc/coreutils.texi (coreutils): Remove this direntry so that `info coreutils` will display the table of contents, rather than the unlikely intended coreutils Multi-call invocation. This also supports lookups like `info coreutils readlink`, but note that only works with the standard info viewer, and not with pinfo for example. doc: mention in more places that -D will create --target-directory * src/install.c (usage): Mention this commonly required functionality in the -D option description. * doc/coreutils.texi (install invocation): Likewise for the --target-directory description. install: only attempt to create a target dir once * src/install.c (main): As an optimization, when calling install_file_in_dir() for each file, only attempt to create the target directory once, as this is invariant over the loop. install: fix relative copies to absolute directory with -D * src/install.c (mkancesdirs_safe_wd): Unconditionally restore the current working directory when possibly called multiple times (from install_file_in_dir()). * tests/install/create-leading.sh: Add a test case. * NEWS: Mention the fix. Fixes http://bugs.gnu.org/21497 2016-01-02 Bernhard Voelker maint: fix sc_long_lines to warn about 100+ maximum line length again Since commit v8.23-60-g414a8cf, the above check failed to find sources where the length of the longest line is a 3 or more digit number. * cfg.mk (sc_long_lines): Remove the '\' escape character before '{' and '}', because this would mean literal '{' and '}' characters in an extended regular expression in sed(1). 2016-01-01 Pádraig Brady maint: avoid a sc_long_lines failure in recent commit * src/dd.c (alloc_ibuf): Shorten line length. 2016-01-01 Pádraig Brady maint: update all copyright year number ranges Run "make update-copyright" and then... * gnulib: Update to latest with copyright year adjusted. * tests/init.sh: Sync with gnulib to pick up copyright year. * bootstrap: Likewise. * tests/sample-test: Adjust to use the single most recent year. 2015-12-31 Paul Eggert build: update gnulib submodule to latest dd: append spaces to shorter status=progress line Problem noted by Pádraig Brady in: http://bugs.gnu.org/22277#8 Also, make the output a bit more precise while we're at it. * NEWS: Document this. * src/dd.c (previous_time): Remove, replacing with ... (next_time): New var. All uses changed. This avoids some rounding errors, and should be a bit faster. (newline_pending): Remove, replacing with ... (progress_len): New var. All uses changed. This lets us keep track of how many trailing spaces to append. (print_xfer_stats): Get the time first thing, so that it's closer to being correct. Count the bytes output, and append trailing spaces if needed. Add remarks to translators about translation lengths. dd: summarize in --human-readable format too Problem reported by Linda Walsh in: http://bugs.gnu.org/17505 * NEWS: Document this. * doc/coreutils.texi (dd invocation): Use a simpler script. Adjust output example to match new behavior. * src/dd.c (human_size): Remove. All uses changed to use human_readable and ... (human_opts): ... this new constant. (abbreviation_lacks_prefix): New function. (print_xfer_stats): Use it. Output both --si and --human-readable summaries, but only if they have prefixes. * tests/dd/reblock.sh, tests/dd/stats.sh: Test new behavior. 2015-12-23 Pádraig Brady build: update to latest gnulib Mainly for these changes: - freadptr: fix to work with ungetc on all uClibc configs - fts: enable leaf optimization for XFS - fts: ensure leaf optimization used for NFS - strftime-tests: avoid false failure on OS X - intprops-tests: avoid warnings (causing CI failures) * NEWS: Update with items from above that are significant from the previous coreutils release. 2015-12-22 Pádraig Brady doc: describe test operator precedence and associativity * doc/coreutils.texi (Connectives for test): Add notes on precedence and associativity. Also mention the portability caveats with these operators. * cfg.mk: Avoid sc_prohibit_test_minus_ao for coreutils.texi. Fixes http://bugs.gnu.org/22216 2015-12-17 Pádraig Brady tests: avoid root test false failure without libxattr * tests/cp/cp-mv-enotsup-xattr.sh: Exclude xattr copy verification when libxattr is not available. 2015-12-17 Bernhard Voelker build: rely on umask for setting permissions in tarball Commit v8.24-116-g896006b changed the file permissions for the tarball using ug+rw. Better to let the umask decide whether the file should be group-writeable or not. * Makefile.am (dist-hook): Change chmod call from 'ug+rw' to '+rw'. 2015-12-17 Pádraig Brady tests: avoid intermittent ulimit -v failures * init.cfg (get_min_ulimit_v_): Increase the determined virtual memory limit for the command by a page size to avoid alignment issues causing false failures for subsequent runs. * tests/misc/cut-huge-range.sh: Be defensive and match the cut invocations under test, more closely with the form used to determine the VM memory limit. This was noticed on i686 linux at: http://hydra.nixos.org/build/28990456 2015-12-17 Pádraig Brady build: ensure all files are writable in the tarball * Makefile.am (dist-hook): Ensure all files are writable so that extracted tarballs are easy to remove, without getting prompts about removing read-only files. Reported by Benno Schulenberg. 2015-12-15 Pádraig Brady maint: ensure emit_mandatory_arg_note() used if needed * cfg.mk (sc_emit_mandatory_arg_note): Add new syntax check. maint: include 'env' commands in returns_ syntax check * cfg.mk (sc_prohibit_and_fail_1): Remove 'env' from the exclusion list. * tests/misc/env.sh: Adjust accordingly. * tests/misc/help-version.sh: Likewise. tests: improve kill tests * tests/misc/kill.sh: - verify kill -[lt] support multiple parameters. - verify kill -[lt] fail with (some) invalid parameters. - Use returns_1 ... || fail=1, rather than ... && fail=1. 2015-12-14 Pádraig Brady sort: fix --debug marking for -b -k1.x We were erroneously skipping blanks in the marked comparison _after_ the key start offset was applied. * src/sort.c (debug_keys): Don't skip starting blanks if already handled by begfield(). * tests/misc/sort-debug-keys.sh: Add a test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/22155 stat: report file system type for "overlayfs" * src/stat.c (human_fstype): Add "overlayfs", and tag it as "remote" to ensure that tail continues to use the more conservative polling mode. * README-release: Remove a stale comment about updating fremote() in tail.c. Also give a link to *_SUPER_MAGIC definitions not in the standard linux/magic.h location. * NEWS: Mention that this file system is recognized. 2015-12-14 Pádraig Brady kill: add undocumented -L for compatibility procps, bash >= 4.4, and util-linux >= 2.23 all have the -L option to output a table. * src/kill.c (main): Have -L synonymous with -t. Luckily there is no signal that begins with 'L'. * tests/misc/kill.sh: Update the test. 2015-12-14 Pádraig Brady tests: add tests for the kill command * tests/misc/kill.sh: Add first tests for kill. * tests/local.mk: Reference the new test. maint: fix incorrect spellings etc * NEWS: Use a standard version specification. * doc/coreutils.texi: Fix incorrect grammar. * src/df.c: Update a stale comment. * src/copy.c: Fix incorrect spellings. * src/factor.c: Likewise. * src/ls.c: Likewise. * src/pr.c: Likewise. * src/relpath.c: Likewise. * src/shred.c: Likewise. * src/sort.c: Likewise. * src/split.c: Likewise. * src/stdbuf.c: Likewise. * tests/misc/seq-precision.sh: Likewise. 2015-12-14 Pádraig Brady maint: remove form feed characters from sources These are no longer useful and can impact viewing patches in some mail programs for example. * src/cksum.c: Remove ^L characters. * src/comm.c: Likewise. * src/kill.c Likewise. * src/ln.c Likewise. * src/nl.c Likewise. * src/pr.c Likewise. * src/ptx.c Likewise. * src/split.c Likewise. * src/tr.c Likewise. * src/tsort.c Likewise. * src/uniq.c Likewise. * src/wc.c Likewise. * cfg.mk (sc_prohibit-form-feed): A new syntax check. 2015-12-07 Eric Blake sort: promote '--debug' * src/sort.c (usage): Mention --debug in KEYDEF paragraph. 2015-12-07 Paul Eggert doc: promote 'sort --debug' * README, doc/coreutils.texi (Introduction, sort invocation): Suggest 'sort --debug' more prominently. 2015-12-04 Pádraig Brady doc: improve the description of sort --random-sort * src/sort.c (usage): Avoid mentioning the implementation, rather focusing on the result of the operation. 2015-12-03 Paul Eggert build: port to Studio C on Solaris 12 Reported by Rich Burridge in: http://bugs.gnu.org/22087 * configure.ac (HAVE_UT_HOST, HAVE_C_LINE, stdbuf): Pacify picky compilers that complain about unreachable statements. 2015-12-01 Bernhard Voelker doc: clarify behaviour of mv for existing target directories * doc/coreutils.texi (mv invocation): Add a sentence to clarify that mv(1) will only replace empty directories in the target, and fail with a diagnostic otherwise. Discussed at: http://lists.gnu.org/archive/html/coreutils/2015-12/msg00000.html 2015-11-30 Bernhard Voelker doc: avoid confusing K argument in the man pages of head and tail The character 'K' representing numeric user input can be confusing, especially in the context where it can be suffixed by the multipliers K, M, G, etc. Use NUM instead. * doc/coreutils.texi (head invocation): Replace @var{k} by @var{num}. Also change @var{count} in the paragraph about the obsolete option syntax. (tail invocation): Likewise. Furthermore, adjust the option order to alphabetical sorting. * src/head.c (usage): s/K/NUM/ in as placeholder in the -c and -n options. Furthermore, print the actual DEFAULT_NUMBER instead of the hard-coded 10. * src/tail.c (usage): s/K/NUM/ in as placeholder in the -c and -n options. Add "[+]" to clarify that NUM may optionally be preceded by this sign. Remove the redundant explanation of the optional '+' prefix for NUM, as that is now clear from the option description above. Suggested by Dan Jacobson in http://bugs.gnu.org/22042 2015-11-28 Bernhard Voelker doc: reference each other in the man pages of head and tail * man/head.x: Add "SEE ALSO" section referencing tail(1). * man/tail.x: Add "SEE ALSO" section referencing head(1). Suggested by Dan Jacobson in http://bugs.gnu.org/22041 2015-11-28 Pádraig Brady build: support reproducible builds with consistent man pages Use a consistent date in the generated man pages even if SOURCE_DATE_EPOCH is not set. * Makefile.am: Generate .timestamp in the tarball. * man/local.mk: Pass .timestamp if available to help2man. * man/dummy-man: Don't bother with the year in the stub. Reported in http://bugs.debian.org/806321 2015-11-27 Pádraig Brady doc: update to latest help2man Sync with version 1.47.3 which provides these significant changes: - support for reproducible builds by using $SOURCE_DATE_EPOCH. - Adjust spacing of italic text at roman/italic boundaries. 2015-11-25 Pádraig Brady tests: for compare_(), use cmp -s where available * tests/init.sh (compare_): Only fall back to cmp without the POSIX defined -s option, where this is not available. build: avoid makeinfo unless required * doc/local.mk (constants.texi): Avoid calling makeinfo unless the constants derived from tail and shred have changed. This avoids a dependence on makeinfo from patched sources, and avoids an expensive makeinfo call for developers. doc: clarify descriptions for tr --squeeze-repeats * doc/coreutils.texi (uniq invocation): Fix the description of problematic input to say "blank lines" rather than "two or more blank lines". (tr invocation): Clarify that -s only works with SET1 when not translating (when SET2 not specified). Also explicitly state in examples where blank lines are deleted. Also add "deleting" to the menu item. * src/tr.c (usage): Improve the -s summary to say it always operates on the last specified SET. doc: provide an example for prefixing all tail output lines * doc/coreutils.texi (tail invocation): Provide an example using awk to convert tail ==> file <== headers to file: prefixes on each line. Suggested by Stephen Shirley. 2015-11-24 Dmitry Monakhov copy: fix copying of extents beyond the apparent file size fallocate can allocate extents beyond EOF via FALLOC_FL_KEEP_SIZE. Where there is a gap (hole) between the extents, and EOF is within that gap, the final hole wasn't reproduced, resulting in silent data corruption in the copied file (size too small). * src/copy.c (extent_copy): Ensure we don't process extents beyond the apparent file size, since processing and allocating those is not currently supported. * tests/cp/fiemap-extents.sh: Renamed from tests/cp/fiemap-empty.sh and re-enable parts checking the extents at and beyond EOF. * tests/local.mk: Reference the renamed test. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/21790 2015-11-23 Pádraig Brady md5sum,sha*sum: ensure --ignore-missing fails when no file verified * src/md5sum.c (digest_check): Update a matched_checksums bool upon matched checksum, and fail (loudly unless --status is specified) if there were no matches. Also change properly_formatted_lines to a bool while at it since we don't need to track the plurality. * tests/misc/md5sum.pl: Add a test case. Suggested by Jim Meyering. 2015-11-23 Luther Thompson md5sum,sha*sum: add --ignore-missing for checking a subset of files * doc/coreutils.texi (md5sum invocation): Document the new option. * src/md5sum.c (digest_file): Return an empty digest to indicate a missing file. (digest_check): Don't fail or output status given an empty checksum. (usage): Document the new option. (main): Process and validate the new option. * tests/misc/md5sum.pl: Add new test cases. * NEWS: Mention the new feature. Fixes http://bugs.gnu.org/15604 2015-11-23 Pádraig Brady maint: allow 'sha*sum:' tag in commit summaries To support "md5sum,sha*sum:" Suggested by Bernhard Voelker. 2015-11-21 Pádraig Brady doc: give a tee example for combining process substitution outputs This can be useful if you want to further process data from process substitutions. For example: datagen | tee >(md5sum --tag) > >(sha256sum --tag) | sort * doc/coreutils.texi (tee invocation): Mention that -p is useful with pipes that may not consume all data. Add an example, similar to the one above. * THANKS.in: Add Jirka Hladky. 2015-11-21 Pádraig Brady tests: avoid false failure on older NFS implementations * tests/ls/stat-failed.sh: Skip the test if 'd' is returned as the type, and document where this was seen. Also flag failure to write small temp files during the test as an error rather than a failure. Fixes http://bugs.gnu.org/21130 2015-11-19 Pádraig Brady build: fix VPATH build with --disable-dependency-tracking * src/local.mk: Run `mkdir -p src` in all our explicit rules, as in a VPATH build the src/ dir is only created at configure time as a side effect of dependency tracking generation. 2015-11-17 Jim Meyering scripts: update versions of gettext and libtool maint: remove unmaintained file, c99-to-c89.diff * src/c99-to-c89.diff: Remove file. * src/local.mk (EXTRA_DIST): Remove it from this list. * README (Pre-C99 build failure): Update section. 2015-11-12 Assaf Gordon csplit: check and report fwrite errors with errno discussed in: http://lists.gnu.org/archive/html/coreutils/2015-10/msg00091.html * src/csplit.c: (save_line_to_file): check fwrite failures, report and exit immediately instead of deferring to 'close_output'. * tests/misc/csplit-io-err.sh: test fwrite failure using LD_PRELOAD. * tests/local.mk: add new test. 2015-11-11 Pádraig Brady stat: improve support for new Linux pseudo FS and ACFS * src/stat.c (human_fstype): Add new file system ID definitions. * NEWS: Mention the improvement. 2015-11-10 Pádraig Brady tests: ensure programs are built before testing programs may not be built due to missing system dependencies, or any program can be excluded at configure time with --enable-no-install-program. So ensure we're not testing the system versions in these cases. * init.cfg (print_ver_): Call require_built_ first. * tests/misc/tty-eof.pl: Skip programs not built. * tests/Coreutils.pm (run_tests): Likewise. * tests/misc/ls-misc.pl: Use 'env test' rather than abs path. * tests/misc/test-diag.pl: Likewise. * tests/local.mk: Adjust include order for dependencies. * tests/misc/arch.sh: Remove redundant calls to require_built_. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/groups-dash.sh: Likewise. * tests/misc/groups-version.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/cp/acl.sh: Remove problematic call to print_ver_ [gs]etfacl. * tests/mv/acl.sh: Likewise. * cfg.mk (sc_env_test_dependencies): A new syntax check to enforce specifying dependencies with print_ver_ for programs specified through the env command. * du/bigtime.sh: Add new print_ver_ dependencies. * du/max-depth.sh: Likewise. * dd/ascii.sh: Likewise. * tests/ls/capability.sh: Likewise. * tests/ls/root-rel-symlink-color.sh: Likewise. * tests/misc/chroot-fail.sh: Likewise. * tests/misc/readlink-fp-loop.sh: Likewise. * tests/misc/sort-debug-keys.sh: Likewise. * tests/readlink/can-e.sh: Likewise. * tests/readlink/can-f.sh: Likewise. * tests/readlink/can-m.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/inotify-race2.sh: Likewise. * tests/touch/no-create-missing.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. * tests/misc/printenv.sh: Tweak to avoid syntax check trigger. * tests/misc/help-version.sh: Likewise. * tests/misc/yes.sh: Likewise. * tests/misc/printf-quote.sh: Use previously unused $prog. * configure.ac (EXTRA_MANS): Add $gl_no_install_prog to the list so that check-x-vs-1 syntax check is satisfied. 2015-11-10 Pádraig Brady maint: use standard spacing in shebang line in tests It's better to be consistent even though spacing is insignificant: http://www.in-ulm.de/~mascheck/various/shebang/#blankrequired 2015-11-10 Pádraig Brady tests: cleanup trapping of signal names A side effect of this cleanup is we no longer depend on our own kill command being built. * init.cfg (require_trap_signame_): A new function to verify that the shell supports specifying traps by signal name. (require_kill_group_): A new function to ensure the shell supports sending a signal to a group. * tests/du/move-dir-while-traversing.sh: Ensure trap supports signal names. * tests/misc/stty-invalid.sh: Likewise. * tests/misc/stty-pairs.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise. * tests/misc/stty.sh: Likewise. * tests/misc/sort-compress.sh: Likewise. Also simplify trap call. * tests/install/trap.sh: Likewise. * tests/misc/timeout.sh: Likewise. * tests/dd/stats.sh: Likewise. Also use default kill command. * tests/misc/timeout-group.sh: Likewise. 2015-11-06 Pádraig Brady tests: fix dirent d_type support verification * init.cfg (require_dirent_d_type_): Don't use df -x to exclude XFS, since this depends on a correct mtab which is brittle and often not correct within chroots. * tests/d_type-check: Check also the d_type of files, which excludes XFS appropriately. Specify all argument and return types to avoid truncated pointers being passed, which skipped the test due to crashes on x86_64 at least. Simplify the C library lookup by reusing the interpreter's. chroot issue reported at https://bugzilla.redhat.com/1263341 2015-11-04 Pádraig Brady maint: add a syntax check to avoid unstyled quoting * src/paste.c (main): Use our styled wrapper for quotearg_colon(). * cfg.mk (sc_prohibit-quotearg): A new syntax check to avoid future uses of unstyled quotearg to one of the internal slots, and thus destined for diagnostic output. paste: avoid confusing backslash quoting in diagnostic * src/paste.c (main): Setting the quoting style to "escape" went against the intent of the comment about presenting doubled backslashes to the user. Instead use "c-maybe" which is the only mode which avoids doubled backslashes, and provides protection against arbitrary control characters. * tests/misc/paste.pl: Adjust accordingly. ls: default to --quoting=shell-escape for output to terminal * src/ls.c (decode_switches): Set "shell-escape" if isatty(). * doc/coreutils.texi (ls invocation): Update the defaults description. * NEWS: Mention the change in behavior. It should not have backwards compat issues, but mentioning here just in case. printf: support the %q format to quote for shell * src/printf.c (usage): Mention the new format. (print_formatted): Handle the quoting by calling out to the quotearg module with "shell-escape" mode. * doc/coreutils.texi (printf invocation): Document %q. * tests/misc/printf-quote.sh: New test. * tests/local.mk: Reference new test. * NEWS: Mention the new feature. 2015-11-04 Pádraig Brady ls: avoid redundant processing when already escaping This is mainly noticeable when the multi-byte code within ls.c is triggered by multi-byte quotes. $ seq 200000 | xargs touch $ time ls-old -U --quoting=locale --hide-control-chars >/dev/null real 0m0.483s $ time ls-new -U --quoting=locale --hide-control-chars >/dev/null real 0m0.430s * src/ls.c (quote_name): Avoid rescanning the output looking for unprintable chars when we know the quoting mode already escapes them. * tests/misc/ls-misc.pl: Add tests for all quoting modes, with and without -q, to verify this assumption. 2015-11-04 Pádraig Brady ls: document and test new shell-escape quoting * doc/coreutils.texi (ls invocation): Describe the new 'shell-escape' and 'shell-escape-always' quoting options. * src/ls.c (usage): Mention the new quoting options. * tests/misc/ls-misc.pl: Add a test for 'shell-escape' test: use consistent quoting * src/test.c (test_syntax_error): Reuse verror() rather than open coding the error output format. (term): Don't hardcode '' quoting. (main): Likewise. 2015-11-04 Pádraig Brady all: avoid quoting file names when possible Quote file names using the "shell-escape" or "shell-escape-always" methods, which quote as appropriate for most shells, and better support copy and paste of presented names. The "always" variant is used when the file name is embedded in an error message with surrounding spaces. * cfg.mk (sc_error_shell_quotes): A new syntax check rule to suggest quotef() where appropriate. (sc_error_shell_always_quotes): Likewise for quoteaf(). * src/system.h (quotef): A new define to apply shell quoting when needed. I.E. when shell character or ':' is present. (quoteaf): Likewise, but always quote. * src/*.c: Use quotef() and quoteaf() rather than quote() where appropriate. * tests/: Adjust accordingly. 2015-11-04 Pádraig Brady build: update gnulib submodule to latest Includes support for "shell-escape" from quotearg md5sum: ensure a single status line per file * src/md5sum.c: Use the same file name escaping method used when generating and checking checksums. I.E. ensure a single line per file by starting the line with '\' for any file name containing '\n' and replacing those with "\\n". * NEWS: Move the item from changes in behavior to improvements, since this is no longer a backwards incompat change when processing stdout status messages. * tests/misc/md5sum.pl: Remove quotes from expected status output. * tests/misc/sha1sum.pl: Likewise. 2015-11-04 Pádraig Brady all: replace most uses of quotearg_colon() with quote() Related to commit v8.24-61-g6796698 this provides more consistent quoting, as quotearg_colon() defaults to "literal" quoting by default, while quote() provides appropriate quoting for diagnostics by default. * gl/modules/randread: Depend on quote module rather than quotearg. * gl/lib/randread.c: Used quote() not quotearg_colon(). * src/: Likewise. * src/shred.c: Likewise. Also avoid unnecessary quoting introducing overhead when wiping names. * cfg.mk: Relax the matching expression to allow "qname" variables as used in shred.c to satisfy the check. * tests/: Adjust accordingly. 2015-11-02 Pádraig Brady doc: fix texinfo for short options taking a parameter * doc/coreutils.texi: (tail invocation): Add missing -s, along with the existing --sleep-interval description. (csplit invocation): s/--suffix/--suffix-format/. (head invocation): Use same variable (COUNT) for -n and --head-count. (seq invocation): Add opindex items for all options. (ptx invocation): Likewise. Fix typo s/--flac-truncation/--flag-truncation/. (touch invocation): State explicitly that -d takes a parameter, which also indicates that an '=' is not to be used for the short option syntax. (ls invocation): Likewise for the -w option. Fixes http://bugs.gnu.org/21809 2015-10-29 Pádraig Brady maint: allow 'all:' tag in commit summaries To tag changes that are user visible and affect all (or many) commands. 2015-10-28 Pádraig Brady doc: clarify iso-8601 formats used by ls and du * doc/coreutils.texi (du invocation): Remove 'like' from mentions of ISO-8601 as the components are individually conformant. (ls invocation): Likewise, except for --time-style=iso for recent files, where the MM-DD component is not a valid ISO-8601 timestamp. 2015-10-27 Pádraig Brady date: use extended format timezone for --iso-8601 * src/date.c (main): Use %:z rather than %z with --iso-8601 as the standard states to consistently use extended format. Note either format can be parsed by date. * tests/misc/date.pl: Adjust accordingly. * doc/coreutils.texi (du invocation): Clarify that "iso" time styles are only similar to ISO-8601. (ls invocation): Likewise. (date invocation): Adjust the comment stating that only --rfc-3339 output can be parsed by date(1). * NEWS: Mention the change in behavior. Reported at http://bugs.debian.org/799479 doc: reference related commands from users(1) and groups(1) * man/users.x: See also getent, who * man/groups.x: See also getent Addresses http://bugs.gnu.org/21735 copy,dd: simplify and optimize NUL bytes detection * src/factor.c: Move LIKELY() definition to... * src/system.h: ...here. (is_nul): Reimplement with a version that doesn't require a sentinel after the buffer, and which calls down to (the system optimized) memcmp. Performance analyzed at http://rusty.ozlabs.org/?p=560 * src/dd.c (alloc_obuf): Simplify the is_nul() call by not needing to write the sentinel. * src/copy.c (sparse_copy): Likewise. (copy_reg): Simplify the buffer allocation by avoiding consideration of the sentinel in the buffer size calculation. 2015-10-27 Pádraig Brady all: quote string arguments in error messages These strings are often file names or other user specified parameters, which can give confusing errors in the presence of unexpected characters for example. * cfg.mk (sc_error_quotes): A new syntax check rule. * src/*.c: Wrap error() string arguments with quote(). * tests/: Adjust accordingly. * NEWS: Mention the improvement. 2015-10-27 Pádraig Brady md5sum: quote all printed file names This is especially significant when using --check with files generated on a windows system, where the \r characters produce corrupted and confusing error messages. This also ensures status messages are output on a single line. * src/md5sum.c: Use quote() for printed file names. * tests/misc/md5sum.pl: Adjust accordingly. * NEWS: Mention the change in behavior. Fixes http://bugs.gnu.org/21757 2015-10-21 Pádraig Brady ls: fix off by one error when determining max display columns * src/ls.c (main): Account for the first column not including a separator when calculating max_idx. * tests/ls/w-option.sh: Add a test case. * NEWS: Mention the bug fix. ls: allow -w0 to mean no limit on line length * src/ls.c (print_with_separator): Renamed from print_with_commas, and parameterized to accept the separator to print. Also fix an edge case where '\n' not printed when the POS variable overflows SIZE_MAX. (print_current_files): Degenerate -x and -C to using the cheaper print_with_separator() in the -w0 case. * doc/coreutils.texi (ls invocation): Document the new feature. * tests/ls/w-option.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/21325 2015-10-19 Pádraig Brady ls: detect terminal color support using glob patterns * src/ls.c (know_term_type): Corresponding to commit v8.24-48-gc249a5a, use fnmatch to inspect the dircolors database. Noticed due to failing tests/ls/color-{dtype-dir,term}.sh tests. tests: adjust recent change to csplit VM limit * tests/misc/csplit-heap.sh: A little more memory is required for the full run case. Noticed with --enable-single-binary. 2015-10-19 Pádraig Brady maint: fixes to support improved sc_tight_scope The gnulib provided sc_tight_scope target was ineffective, as it was checking against an invalid blank regular expression, and thus ignoring any extern function issues. This is now fixed up in gnulib, and so we need to fix our scoping issues before the next gnulib update. * cfg.mk: Setup and document the tight_scope config variables appropriately. * src/selinux.h: Since declared in *_SOURCES, use the two line form for the extern function declarations. * src/set-fields.h: Add the extern declarations, and since declared in noinst_HEADERS use the single line form. 2015-10-19 Pádraig Brady tests: avoid failure when auto selecting factor tests * tests/factor/run.sh: If this template is found through `grep -El "print_ver_.* factor"` for example, then just skip it. 2015-10-19 Pádraig Brady factor: remove unreachable SQUFOF code at compile time It was a little confusing as to whether the SQUFOF algorithm was enabled, and in fact there were no options available to enable it. Therefore clarify the 3 configurable behaviors for the code to 3 defines at the top of the program, and only include the SQUFOF code if enabled at compile time. $ size src/factor-before text data bss 93997 1412 2504 $ size src/factor-after text data bss 87885 1404 2504 * src/factor.c: Only include the SQUFOF factor code when enabled via the USE_SQUFOF define. * doc/coreutils.texi (factor invocation): Update note about factor limits, as we can factor 128 bit numbers without GMP. 2015-10-19 Pádraig Brady doc: reference shuf(1) from the sort (-R) man page * man/sort.x: Cross reference with shuf(1). * src/sort.c (usage): Mention shuf(1) with -R option. Suggested in http://bugs.debian.org/641166 2015-10-18 Bernhard Voelker maint: avoid duplicate test runs * tests/local.mk (all_tests): Remove the tests 'tests/id/setgid.sh' and 'tests/mkdir/smack-root.sh' because they are mentioned in the 'all_root_tests' list; these tests are skipped anyway during a non-root run because flagged with 'require_root_'. 2015-10-16 Paul Eggert doc: remove obsolete performance comment sha512sum can be faster than sha256sum. E.g., ‘dd if=/dev/zero bs=1024k count=1024 | time sha256sum’ reports 8.16 user CPU seconds on my host, whereas sha512sum consumes 5.45 seconds (Fedora x86-64 on an AMD Phenom II X4 910e). Although sha512sum is still considerably slower on x86, a good chunk of uses are on 64-bit hosts and anyway there’s little point to scaring people away from sha512sum nowadays. * doc/coreutils.texi (sha2 utilities): Remove obsolete comment. 2015-10-16 Jim Meyering maint: avoid uniq.c warning from bleeding-edge gcc's -Wstrict-overflow * src/uniq.c (main): Make the type of "nfiles" unsigned, to avoid a brand new warning from a gcc I built from today's sources (gcc version 6.0.0 20151015 (experimental) (GCC)): src/uniq.c:523:14: error: assuming signed overflow does not occur \ when simplifying conditional to constant [-Werror=strict-overflow] if (nfiles == 2) ^ 2015-10-16 Pádraig Brady dircolors: support globbing of TERM entries * src/dircolors.c (dc_parse_stream): Support globbing of TERM entries, to allow entries like "TERM *256color*" for example. * src/dircolors.hin: Reduce the internal list with globbing. * tests/misc/dircolors.pl: New test cases. * NEWS: Mention the improvement. 2015-10-15 Pádraig Brady tests: avoid false failure in rm/r-root.sh with gdb warnings * tests/rm/r-root.sh: Skip the test if there are gdb warnings that will impact further stderr checks. For example some buggy gdb versions may report "Got object file from memory but can't read symbols: File truncated". Also fix an incorrect stderr check from the previous change. Reported by Bernhard Voelker. tests: avoid false failure in rm/r-root.sh under load * tests/rm/r-root.sh: Use gdb rather than timeout(1) as the last resort protection against unlinkat() calls. The timeout of 2s was susceptible to false positives under load, and gdb is stronger protection in any case. We remove the "expensive" tag on this test also since it should be robust. Reported by Jim Meyering. tests: avoid false failure in a tail test under load * tests/tail-2/follow-stdin.sh: Use the standard tail testing framework to avoid the race seen under very high load, and also test the non inotify case. Reported by Jim Meyering 2015-10-14 Pádraig Brady tests: avoid false failures with default ACLs To reproduce: setfacl -dm group::rx . setfacl -dm other::rx . make check * init.cfg (require_no_default_acl_): A new function to skip when default ACLs are detected, or if the getfacl utility is not available then skip if any non LSM ACLs detected. * tests/cp/existing-perm-race.sh: Call require_no_default_acl_. * tests/mkdir/parents.sh: Likewise. * tests/mkdir/perm.sh: Likewise. 2015-10-12 Pádraig Brady tail: no longer warn about unrecognized file systems * src/tail.c (fremote): No longer prompt the user to email with the unrecognized file system constant, since we have process in place to sync periodically with the latest Linux constants, and the fall back polling mode is still fully functional. 2015-10-10 Pádraig Brady tests: avoid false failure when restorecon is ineffective * tests/cp/cp-a-selinux.sh: Ensure we skip the portion of the test depending on restorecon to be effective. I.E. also skip when restorecon warns, as it doesn't exit with error status when matchpathcon fails to find a match for a file. This is the case in /tmp on Fedora for example, in which case the new destination that cp creates will have the default security context of the process, rather than the explicit context we set on the source file. Details at: http://bugzilla.redhat.com/1247641 tests: make a long running test responsive to Ctrl-C * tests/misc/sort-compress-hang.sh: Use --foreground with the timeout(1) command (noting the caveats), to run the sort command in the foreground program group, and thus be responsive to Ctrl-C. This very_expensive_ test takes over a minute on a i3-2310M, with RAM backed /tmp. 2015-10-08 Pádraig Brady tests: adjust recent changes to virtual memory limits * tests/dd/no-allocate.sh: Account for timeout(1) when determining the required mem, as timeout has additional shared libs. This avoids the need for the hardcoded 4M addition to the limit. * tests/misc/head-c.sh: Increase the base limit, to account for the fact that head(1) will allocate some additional mem in this case. * tests/misc/cut-huge-range.sh: Remove mention of specific limits. * tests/misc/printf-surprise.sh: Likewise. Reported by Dmitry V. Levin 2015-10-06 Pádraig Brady csplit: remove erroneous mention of -m in --help * src/csplit.c (usage): -m is not accepted, only --suppress-matched. * tests/misc/csplit-suppress-matched.pl: Spelling fix. Reported by Ondrej Oprala 2015-10-04 Paul Eggert tee: simplify argv handling * src/tee.c (tee_files): Last arg is now char ** instead of char const **, as that is a bit simpler. All callers changed. Modify files[-1], not files[nfiles], as that is a bit faster and simpler. Latter problem pointed out by Rainer Deyke in: http://bugs.gnu.org/21611 2015-10-03 Paul Eggert build: update gnulib submodule to latest build: Pacify GCC 5 on 32-bit hosts This pacifies GCC 5 in a better way, without disabling diagnostics. * src/df.c (main): Tell compiler that optind is positive. * src/shred.c (known): New function. (dopass): Go back to off_t for file sizes. Avoid integer overflow if we run off the end of the file. Tell compiler that a write cannot write more bytes than requested. 2015-10-03 Pádraig Brady tail: handle kernel dentry unlink race Avoid the intermittent loss of "... has become inaccessible" messages. That would cause tests/tail-2/assert.sh to fail sometimes, mainly on uniprocessor systems. * src/tail.c (tail_forever_inotify): Also monitor IN_DELETE events on the directory, to avoid a dentry unlink()..open() race, where the open() on the deleted file was seen to succeed after an, unlink() and a subsequent IN_ATTRIB, was sent to tail. Note an IN_ATTRIB is sent on the monitored file to indicate the change in number of links, and we can't just use a decrease in the number of links to determine the file being unlinked, due to the possibility of the file having multiple links. Reported by Assaf Gordon and Ludovic Courtès. Fixes http://bugs.gnu.org/21460 2015-10-03 Pádraig Brady build: avoid -Wstrict-overflow warnings with GCC 5.1 on 32 bit * src/shred.c (dopass): With -O2, GCC 5.1 gives "assuming signed overflow does not occur when simplifying conditional to constant" warnings, in regard to the signed (off_t) variables. Therefore use unsigned (uintmax_t) instead, and a separate boolean to cater for the special meaning of the negative part of the integer range. Noticed at http://hydra.nixos.org/build/24983447 build: update gnulib submodule to latest Includes a change to xalloc.h to avoid -Wstrict-overflow warnings with GCC 5.1 on 32 bit with optimization enabled. A subsequent commit will fix similar issues in shred. 2015-10-02 Dario Giovannetti dircolors: add xterm-termite entry * src/dircolors.hin: Add "xterm-termite" as this VTE based terminal emulator is quite different from xterm, despite the name. For example "Termite supports italic text and it won't work if TERM is set to xterm. Even the backspace key won't work properly anymore for applications relying on terminfo". Reported also by Lukas Sabota and Sven-Hendrik Haase. 2015-09-23 Dave Chiluk df: prioritize mounts nearer the device root In the presence of bind mounts of a device, the 4th "mount root" field from /proc/self/mountinfo is now considered, so as to prefer mount points closer to the root of the device. Note on older systems with an /etc/mtab file, the source device was listed as the originating directory, and so this was not an issue. Details at http://pad.lv/1432871 * src/df.c (filter_mount_list): When deduplicating mount entries, only prefer sources nearer or at the root of the device, when the target is nearer the root of the device. * NEWS: Mention the change in behavior. 2015-09-23 Pádraig Brady build: update gnulib submodule to latest This includes a change to propagate the 4th "mount root" field from /proc/self/mountinfo from the mountlist module, which is needed in a subsequent commit in df. * gl/lib/regcomp.c.diff: Regenerate against latest gnulib. * gl/lib/regex_internal.c.diff: Likewise. * gl/lib/regex_internal.h.diff: Likewise. * cfg.mk: Exclude diffs from trailing whitespace check, which is generally correct, and now needed. 2015-09-22 Bernhard Voelker maint: use adaptive approach for `ulimit -v` based tests When configured with either 'symlinks' or 'shebangs' as value for the --enable-single-binary option, tests based on `ulimit -v` are skipped. The reason is that the multicall 'coreutils' binary requires much more memory due to shared libraries being loaded, and the size of the 'date' binary (~290KiB) compared to the multicall binary (~5MiB), of course. Finally, in the case of 'shebangs', the starting shell requires more memory, too Instead of using hard-coded values for the memory limit, use an adaptive approach: first determine the amount of memory for a similar, yet more trivial invocation of the command, and then do the real test run using that limit (plus some buffer in some cases). * init.cfg (require_ulimit_v_): Remove function. (get_min_ulimit_v_): Add function to determine the minimum memory limit required for a given command in an adaptive way. * cfg.mk (sc_prohibit_test_ulimit_without_require_): Change the name of the above function in the syntax-check rule. * tests/cp/link-heap.sh: Use the above function to determine the minimum memory required to run a command simpler than in the real test run. Use that limit plus a buffer there. While at it, change to list of commands in the subshell to fail also if the beginning `ulimit -v` fails. * tests/dd/no-allocate.sh: Likewise. * tests/misc/csplit-heap.sh: Likewise. * tests/misc/cut-huge-range.sh: Likewise. * tests/misc/head-c.sh: Likewise. * tests/misc/printf-surprise.sh: Likewise. * tests/split/line-bytes.sh: Likewise. * tests/rm/many-dir-entries-vs-OOM.sh: Likewise - doing it separately for each program under test. 2015-09-22 Pádraig Brady maint: avoid deprecation warning with * src/runcon.c (main): As per the compile time warning from libselinux-2.4-3, lookup the class with string_to_security_class(), rather than using defines from flask.h. sort,numfmt: with --debug, diagnose failure to set locale * src/sort.c (main): With --debug, warn upon setlocale() failure, which can happen due to incorrectly specified environment variables, or due to memory exhaustion (simulated with ulimit -v), etc. * tests/misc/sort-debug-warn.sh: Add a test case. See also http://savannah.gnu.org/bugs/11004 2015-09-18 Bernhard Voelker du: avoid to stat all mount points if possible du calls stat for each mount point at startup. This would block or even make du fail if stat for an unrelated mount point hangs. The result is not needed in the normal case anyway and therefore should be avoided. Issue introduced in commit v8.19-2-gcf7e1b5. * src/du.c (fill_mount_table): Move function up as it's not used ... (mount_point_in_fts_cycle): ... here, i.e., the DI_MNT set is initialized and filled only iff FTS has detected a directory cycle. (main): Remove the initialization and filling of the DI_MNT set here, and free the DI_MNT set only if it was used. 2015-09-17 Paul Eggert shred: don’t document -NUMBER option The -NUMBER option was removed from ‘shred’ in 1999, but the manual wasn’t updated to match. Problem reported by Nick Rose in: http://bugs.gnu.org/21502 * doc/coreutils.texi (shred invocation): Remove documentation for -NUMBER option. 2015-09-12 Assaf Gordon numfmt: use new set-fields module to parse --field numfmt --field=LIST can accept the same options as cut. * bootstrap.conf: remove xlist, linked-list * src/local.mk: link numfmt with set-fields * src/numfmt.c: use set-fields.c instead of custom field parsing code. (include_field): adapt to new code. * tests/misc/numfmt.pl: add new tests, adapt current tests to new error message wording from set-fields.c 2015-09-12 Assaf Gordon cut: refactor into set-fields module Extract the functionality of parsing --field=LIST into a separate module, to be used by other programs. * src/cut.c: move field parsing code from here ... * src/set-fields.{c,h}: ... to here. (set_fields): generalize by supporting multiple parsing/reporting options. (struct range_pair): rename to field_range_pair. * src/local.mk: link cut with set-field. * po/POTFILES.in: add set-field.c * tests/misc/cut.pl: update wording of error messages 2015-09-08 Pádraig Brady maint: fix heap manipulations in previous commit * src/sort.c (main): Ensure we don't free() and invalid pointer when reading implicit stdin. Also avoid "definitely lost" valgrind warnings in the --files0-from case. 2015-09-04 Pádraig Brady maint: avoid "definitely lost" valgrind warnings Since commit v8.23-19-g8defcee, main() will return, rather than call exit(), this inducing "definitely lost" warnings in valgrind's leak checker. That precludes using the following to flag memory leaks: valgrind --leak-check=full --error-exitcode=1 \ --errors-for-leak-kinds=definite * src/pr.c (main): In dev builds, explicitly free memory allocated. * src/sort.c (main): Likewise. * src/tail.c (main): Likewise. * src/tsort.c (tsort): Likewise. 2015-09-03 Pádraig Brady tests: test numfmt stdin behavior * tests/misc/tty-eof.pl: Add numfmt to the list of programs that accept input on stdin. ls,ptx: restrict quotearg use to file name output * src/ls.c (getenv_quoting_style, decode_switches, parse_ls_color): Use quote() rather than quotearg(), as the latter defaults to outputting the input unquoted. * src/ptx.c (main): Likewise. base64: no longer support hex or oct --wrap params * src/base64.c (main): Support decimal numbers with leading zeros, by disabling the auto detection of octal and hex. It's not envisaged that base conversion is needed for --wrap parameters, and in the edge case it is, $((0x0)) shell constructs can be used. * tests/misc/base64.pl: Adjust accordingly. * NEWS: Mention the change in behavior. base64: use stricter validation on wrap column * src/base64.c (main): Use the higher level xnumtoumax() rather than xstrtoumax(), which is simpler and improves validation of input. Also pass the _empty_ rather than NULL string as the suffixes parameter so that invalid trailing characters are not allowed. For example -w08 is now flagged as an error, rather than being interpreted as 0. A subsequent commit will further improve verification of numbers with leading zeros by dropping backwards compatibility wrt auto parsing oct and hex numbers. * tests/misc/base64.pl: Add tests for invalid wrap values. 2015-09-03 Pádraig Brady base32: A new program similar to base64 Suggested in https://bugzilla.redhat.com/1250113 * AUTHORS: Add base32. * THANKS.in: Add suggester. * README: Reference the new program. * NEWS: Mention the new program. * src/.gitignore: Ignore the new binary. * bootstrap.conf: Reference the gnulib base32 module. * build-aux/gen-lists-of-programs.sh: Add base32. * man/base32.x: A new template. * man/.gitignore: Ignore the new man page. * man/local.mk: Reference the new man page. * doc/coreutils.texi (base32 invocation): Document the new command. * src/local.mk: Adjust to build base32 based on base64.c. * src/base64.c: Parameterize to use the correct headers, functions and buffer sizes, depending on which binary is being built. * tests/misc/base64.pl: Adjust to test both base32 and base64. * tests/misc/tty-eof.pl: Add base32 as a program that accepts input on stdin without any options specified. * scripts/git-hooks/commit-msg: Add base32 to the template. 2015-09-01 Pádraig Brady build: update gnulib submodule to latest This includes a tweak to support building the gnulib base32 module with -Wsuggest-attribute=const 2015-08-31 Pádraig Brady doc: clarify in --help/man where short options take no param * src/shred.c (usage): For -u, separate the decscription of the short and long option, to clarify that the short option takes no parameter. * src/split.c (usage): Likewise for -d. * src/tee.c (usage): Likewise for -p. * src/uniq.c (usage): Likewise for -D. Suggested by Stephane Chazelas 2015-08-30 Bernhard Voelker tests: avoid FP of ls/stat-free-color.sh with newer glibc Since glibc-2.22, specifically commit [0], the opendir() implementation implicitly makes an additional stat call thus leading to a FP. Seen on openSUSE:Tumbleweed since snapshot 20150821. [0] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=46f894d8c60a * tests/ls/stat-free-color.sh: Change the test to verify that ls(1) needs the same number of stat-like calls for a single, empty directory argument as for one with a few directory entries (sub-directory, regular file, symlink, etc.). 2015-08-27 Pádraig Brady doc: clarify where ambiguous if short options take no param * doc/coreutils.texi (split invocation): Clarify that -d takes no param. (uniq invocation): Likewise for -D. (shred invocation): Likewise for -u. (tee invocation): Likewise for -p. 2015-08-23 Paul Eggert ls: allow -w18446744073709551616 Problem reported by Beco in: http://bugs.gnu.org/21325 * src/ls.c (set_line_length): New function. (decode_switches): Use it to decode COLUMNS and -w. 2015-08-18 Bernhard Voelker maint: add syntax check to ensure larger man/*.x files have a Copyright * cfg.mk (sc_man_check_x_copyright): Add rule to ensure that non-trivial .x files in the 'man/' subdirectory, i.e., files exceeding a line count of 20 or a byte count of 1000, contain a proper Copyright notice. 2015-07-24 Paul Eggert build: fprintftime/nstrftime API changes * bootstrap.conf (gnulib_modules): Add time_rz, since the main source code now uses timezone_t. * src/date.c (batch_convert, main, show_date): * src/ls.c (align_nstrftime, long_time_expected_width) (print_long_format): * src/stat.c (human_time): Use timezone_t rather than boolean to specify which time zone is wanted. * src/ls.c (localtz): New static var. (main): Initialize it. build: update gnulib submodule to latest 2015-07-22 Pádraig Brady doc: discourage use of uname -i and -p options * src/uname.c (usage): State that the non POSIX -i and -p options are non-portable. * doc/coreutils.texi (uname invocation): Mention the discrepancies even across GNU/Linux distros, and that the results should be used as informational only, rather than impacting any logic decisions. Fixes http://bugs.gnu.org/13001 2015-07-15 Pádraig Brady doc: improve man page for realpath -m * src/realpath.c (usage): Mention 'directory' in the --help output, so that ENOTDIR errors may be more easily investigated, by inspecting the man page. Reported at http://pad.lv/1474519 2015-07-15 Peter Bray tests: avoid test warning with perl >= 5.22 * doc/local.mk (sc-lower-case-var): Escape a literal left curly bracket, needed with perl >= 5.22 Fixes http://bugs.gnu.org/21060 sync: fix build with separate $(LIB_FDATASYNC) * src/local.mk: Link with $(LIB_FDATASYNC) needed on some Solaris 10 systems for example. Fixes http://bugs.gnu.org/21059 2015-07-09 Pádraig Brady dircolors: add tmux entries * src/dircolors.hin: tmux entries were added to ncurses in: http://ncurses.scripts.mit.edu/?p=ncurses.git;a=blobdiff;\ f=misc/terminfo.src;h=ce9bdc3b;hp=7e576ef1;hb=be512fa0;hpb=ee1bcda2 2015-07-07 Pádraig Brady shred: fix pattern selection for certain iteration counts This was detected in about 25% of runs with gcc -fsanitize=address ERROR: AddressSanitizer: global-buffer-overflow on address ... READ of size 4 at 0x000000416628 thread T0 #0 0x40479f in genpattern src/shred.c:782 #1 0x4050d9 in do_wipefd src/shred.c:921 #2 0x406203 in wipefile src/shred.c:1175 #3 0x406b84 in main src/shred.c:1316 #4 0x7f3454a1ef9f in __libc_start_main (/lib64/libc.so.6+0x1ff9f) #5 0x4025d8 (/tmp/coreutils-8.23/src/shred+0x4025d8) 0x000000416628 is located 56 bytes to the left of global variable '*.LC49' from 'src/shred.c' (0x416660) of size 17 0x000000416628 is located 12 bytes to the right of global variable 'patterns' from 'src/shred.c' (0x416540) of size 220 SUMMARY: AddressSanitizer: global-buffer-overflow src/shred.c:782 * src/shred.c (gen_patterns): Restrict pattern selection to the K available, which regressed due to v5.92-1462-g65533e1. * tests/misc/shred-passes.sh: Add a deterministic test case. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/20998 2015-07-03 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.24 * NEWS: Record release date. maint: fix distclean-check failure due to THANKS * Makefile.am: Remove our dependence on src/sort which induces awkward dependencies for `make dist` since THANKS will be rebuilt once src/sort is newer. Instead we remove the problematic -f option to sort which actually doesn't change the order given our current input. maint: avoid false syntax-check failure in distcheck * cfk.mk (sc_tests_executable): Restrict the check to git files, so we don't flag any gnulib files added to test/ during `make distcheck`. tests: update gnulib submodule and tests/init.sh to latest * gnulib: Update to latest, with fixes to tests edge cases. * tests/init.sh: Update from gnulib. 2015-07-03 Pádraig Brady tests: avoid side effects of $SHELL environment variable Since non interactive shells don't generally set $SHELL, its value is propagated through the tests and may cause issues; for example if $SHELL implicitly adjusts $PATH when run. Instead we set $SHELL to that determined by the posix-shell module, and use that consistently for all test sub scripts, including those created thorugh the `split --filter` command. * tests/local.mk: Explicitly set $SHELL to $(PREFERABLY_POSIX_SHELL) which defaults to $CONFIG_SHELL and thus usually /bin/sh. * tests/envvar-check: Remove bash environment variables with side effects, in case /bin/bash was selected for $SHELL. * tests/misc/help-version.sh: Remove redundant initialization of $SHELL. * tests/install/strip-program.sh: Use $SHELL for sub script. * tests/misc/sort-compress-hang.sh: Likewise. * tests/misc/sort-compress-proc.sh: Likewise. * tests/misc/sort-compress.sh: Likewise. * tests/misc/timeout-group.sh: Likewise. * tests/rm/fail-eperm.xpl: Remove redundant elision of bash env vars. * tests/misc/pwd-long.sh: Likewise. 2015-07-02 Pádraig Brady tests: avoid false failures on OpenBSD 5.7 * tests/du/threshold.sh: Homogenize getopt error messages. * tests/misc/numfmt.pl: Likewise. * tests/mv/i-3.sh: Skip on *BSD not just FreeBSD. 2015-07-02 Pádraig Brady wc: fix reading of /proc files on aarch64 tests/misc/wc-proc.sh fails when the page size is 64K * src/wc.c (wc): The lseek adjustment should be based on st_blksize, rather than on the internal buffer size. This is significant on aarch64 where st_blksize in /proc is the 64K (the page size) and thus larger than the internal buffer. * src/split.c (main): Even though the similar processing is done on the internal buffer size, that's based on st_blksize and so fine in this regard. Add an assert to enforce this. Avoid this path for the undocumented ---io-blksize option. 2015-07-01 Pádraig Brady build: update gnulib submodule to latest Mainly with build fixes for FreeBSD and OS X. numfmt: increase precision on 32 bit FreeBSD * m4/jm-macros.m4 (HAVE_FPSETPREC): Define if needed. * src/numfmt.c (main): Call fpsetprec() if needed. Fixes large-15 and large-16 test failures on 32 bit FreeBSD. tests: avoid false failure on FreeBSD systems * tests/misc/stty.sh: FreeBSD returns ENOTTY for the TIOCEXT ioctl, so just avoid this option for now. 2015-07-01 Pádraig Brady factor: ensure atomic output through pipes The new tests/misc/factor-parallel.sh test was seen to fail on FreeBSD (derived) systems, which was due to split(1) --filter reading partial lines through pipes, as factor(1) was writing a little over PIPE_BUF each time. * src/factor.c (lbuf): A new structure to internally buffer lines. (lbuf_alloc): A new function to allocate enough at program start. (lbuf_putint): A new function to buffer a uintmax_t. (lbuf_flush): A new function to write directly to standard output. (lbuf_putc): A new function to buffer a character and if enough lines are buffered, then output complete lines <= PIPE_BUF, and continue to buffer the rest. (main): Call the internal buffer allocator, and register the final flush from the internal buffer at program exit. 2015-07-01 Pádraig Brady tests: fix false failure on slower systems * tests/dd/stats.sh: Wait 20s for dd to write 250MB through a fifo, rather than 10s for 500MB. The failure was seen often on a lightly loaded SPARC-Enterprise-T5220 running Solaris 10. 2015-06-29 Jim Meyering maint: stdbuf.c: avoid the OS X putenv function * bootstrap.conf (gnulib_modules): Add setenv, to make this module dependency explicit; setenv is also used by split. * src/stdbuf.c (set_LD_PRELOAD) [__APPLE__]: Use the OS X setenv function, rather than putenv, per that documentation: https://developer.apple.com/\ library/mac/documentation/Darwin/Reference/ManPages/man3/putenv.3.html 2015-06-29 Jim Meyering build: numfmt.c: avoid a shadowing warning * src/numfmt.c (parse_field_arg): Rename parameter s/optarg/arg/, to avoid shadowing getopt's global variable. Otherwise, building on OS X, with --enable-gcc-warnings, I saw this: In file included from src/numfmt.c:19:0: src/numfmt.c: In function 'parse_field_arg': ./lib/config.h:3109:25: error: declaration of 'rpl_optarg' shadows\ a global declaration [-Werror=shadow] 2015-06-29 Assaf Gordon numfmt: fix printf argument order * src/numfmt.c (double_to_human): Fix the argument order passed to snprintf, which happened to work on amd64 with its separate va_arg storage area for floats¹, but would fail tests for example on i686. ¹ https://blog.nelhage.com/2010/10/amd64-and-va_arg/ 2015-06-28 Pádraig Brady maint: clarify df's use of device IDs from /proc/self/mountinfo * src/df.c (filter_mount_list): Clarify why we still stat even though devices IDs may already be available. Note using /proc/self/mountinfo is still an advantage to get filtered items with accurate device patchs in chroots and with bind mounts. I.E. on older setups with static /etc/mtab, df will now bypass that to get the more accuracte and dynamic info. 2015-06-26 Pádraig Brady chroot: quote argument in error diagnostic * src/chroot.c (main): Quote the passed argument, to avoid confusing error messages. 2015-06-25 Pádraig Brady maint: clarify integer operations in recent commit * src/factor.c (print_uintmaxes): Comment that the value of n_out doesn't matter on error, and add an explicit cast to avoid any future warnings. Suggested by Jim Meyering RE commit v8.23-229-g4d2d6c5 tests: avoid false failure when running as root * tests/misc/sync.sh: Ensure dir is unreadable before including the permission check. 2015-06-25 Pádraig Brady tests: avoid false failures with LD_PRELOAD=libasan.so.2 The LD_PRELOAD checks by -fsanitize=address are overly strict: https://groups.google.com/forum/#!topic/address-sanitizer/jEvOJgkDqQk A workaround is to first export LD_PRELOAD=libasan.so.2 The tests below are adjusted so that workaround is not discarded. * tests/cp/no-ctx.sh: Append to $LD_PRELOAD. * tests/df/no-mtab-status.sh: Likewise. * tests/df/skip-duplicates.sh: Likewise. * tests/ls/getxattr-speedup.sh: Likewise. * tests/rm/r-root.sh: Likewise. * tests/cp/nfs-removal-race.sh: Likewise. Also check that LD_PRELOAD is effective to aid future maintainability and avoid false failure if libasan.so.2 is not preloaded. 2015-06-25 Pádraig Brady build: update to latest gnulib Mainly for -fsanitize=address and -fsanitize=undefined fixes build: allow build to complete with -fsanitize=address * src/make-prime-list.c (main): When building with the above option, avoid this build stopping error: "LeakSanitizer: detected memory leaks" 2015-06-24 Pádraig Brady maint: avoid undefined behavior in qsort call GCC 5.1.1 -fsanitize=undefined with glibc 2.21 is returning: "runtime error: null pointer passed as argument 1, which is declared to never be null" * src/ptx.c (sort_found_occurs): Avoid the call with no entries. factor: avoid interspersed lines for parallel runs * src/factor.c (n_out): A new global variable to track how much data has been written to stdout. (print_factors_single): Use n_out to determine whether to flush the current (and previous) lines. * tests/misc/factor-parallel.sh: Add a new test. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. seq: handle exponents more consistently src/seq.c (scan_arg): Set precision and width _after_ exponentiation. For example, this will make '1.1e1 12' and '11 1.2e1' equivalent. One can still set the precision by specifying extra precision on the start value, or more naturally with a precision on a step value. * tests/misc/seq-precision.sh: Add new cases. seq: use consistent output format with hex integers * src/seq.c (scan_arg): Set precision to 0 for hex constants (while avoiding hex floats). This will use then use the fast path for these arguments. Note we also set the precision of inf to 0 here, which ensures we use consistent precision on output where possible. * tests/misc/seq-precision.sh: Add corresponding test cases. seq: support inf last item more generally/efficiently * src/seq.c (main): Call seq_fast for infinite last value. This implicitly avoids format conversion on the 999999 -> 1000000 transition. * src/seq.c (seq_fast): Generalize the buffer handling, and adjust to handle the "inf" last value specifics. * tests/misc/seq-precision.sh: A new test. * tests/local.mk: Reference the new test. 2015-06-22 Pádraig Brady doc: list numfmt in the main menu with "Numeric operations" * doc/coreutils.texi (main menu): Add numfmt. numfmt: don't hardcode floating point limits * src/numfmt.c (MAX_UNSCALED_DIGITS): Set this to LDBL_DIG rather than hardcoding at 18 for better portability. * tests/misc/numfmt.pl: Restrict limit tests to supported platforms. numfmt: handle leading zeros correctly * src/numfmt.c (simple_strtod_int): Don't count leading zeros as significant digits. Also have leading zeros as optional for floating point numbers. * tests/misc/numfmt.pl: Add test cases. * NEWS: Mention the fix. 2015-06-22 Pádraig Brady numfmt: avoid integer overflow when rounding Due to existing limits this is usually triggered with an increased precision. We also add further restrictions to the output of increased precision numbers. * src/numfmt.c (simple_round): Avoid intmax_t overflow. (simple_strtod_int): Count digits consistently for precision loss and overflow detection. (prepare_padded_number): Include the precision when excluding numbers to output, since the precision determines the ultimate values used in the rounding scheme in double_to_human(). * tests/misc/numfmt.pl: Add previously failing test cases. * NEWS: Mention the fix. 2015-06-21 Pádraig Brady numfmt: support user specified output precision * src/numfmt.c (usage): Update the --format description to indicate precision is allowed. (parse_format_string): Parse a precision specification like the standard printf does. (double_to_human): Honor the precision in --to mode. * tests/misc/numfmt.pl: New tests. * doc/coreutils.texi (numfmt invocation): Mention the new feature. * NEWS: Likewise. 2015-06-19 Dylan Cali numfmt: implement support for field ranges * src/numfmt.c: Replace field handling code with logic that understands field range specifiers. Instead of processing a single field and printing line prefix/suffix around it, process each field in the line checking whether it has been included for conversion. If so convert and print, otherwise just print the unaltered field. (extract_fields): Removed. (skip_fields): Removed. (process_line): Gutted and heavily reworked. (process_suffixed_number): FIELD is now passed as an arg instead of using a global. (parse_field_arg): New function that parses field range specifiers. (next_field): New function that returns pointers to the next field in a line. (process_field): New function that wraps the field conversion logic (include_field): New function that checks whether a field should be converted (compare_field): New function used for field value comparisons in a gl_list. (free_field): New function used for freeing field values in a gl_list. Global variable FIELD removed. New global variable all_fields indicates whether all fields should be processed. New global variable all_fields_after stores the first field of a N- style range. New global variable all_fields_before stores the last field of a -M style range. New global variable field_list stores explicitly specified fields to process (N N,M or N-M style specifiers). (usage): Document newly supported field range specifiers. * bootstrap.conf: Include xlist and linked-list modules. numfmt now uses the gl_linked_list implementation to store the field ranges. * tests/misc/numfmt.pl: Add tests for 'cut style' field ranges. Adjust existing tests as partial output can occur before an error Remove test for the 'invalid' field -5.. this is now a valid range. * gnulib: update to avoid compiler warnings in linked-list. * NEWS: Mention the new feature. 2015-06-19 Pádraig Brady doc: use correct units in df | numfmt example * src/numfmt.c (usage): Don't scale output from df so that numfmt outputs the correct values. numfmt: handle suffixes consistently with --{from,to}-unit * src/numfmt.c (unit_to_umax): Support SI (power of 10) suffixes with the --from-unit and --to-unit options. Treat suffixes like is done with --from=auto, which for example will change the meaning of --to-unit=G to that of --to-unit=Gi. The suffix support was previously undocumented and it's better to avoid the traditional coreutils suffix handling in numfmt by default. * doc/coreutils.texi: Document the new behavior. Also fix a typo mentioning {from,to}=units=. * tests/misc/numfmt.pl: Adjust accordingly. * NEWS: Mention the change in behavior. 2015-06-17 Pádraig Brady maint: remove stale online manual items at release * gnulib: Update to get the new gnu-web-doc-update with --mirror option. * README-release: Use the --mirror option in the instructions. Also clarify and update various release steps. 2015-06-09 Pádraig Brady tail: display consistent diagnostics upon file replacement * src/tail.c (recheck): Display diagnostices for replaced files even with reused inodes which is a common case. * tests/tail-2/F-vs-missing.sh: Use correct diagnostic in comment. * tests/tail-2/F-vs-rename.sh: Likewise. tail: display file headers correctly with inotify * src/tail.c (tail_forever_inotify): Use the fspec pointer to distinguish previously output files, rather than a descriptor from the inotify event. That event descriptor was that of the parent directory when files were created or renamed etc. (check_fspec): Adjust for the new comparison. Also show the header when the file is truncated, since we show data in this case also. * tests/tail-2/F-headers.sh: A new test case. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. 2015-06-07 Pádraig Brady maint: sync .gitignore items with gnulib entries * .gitignore: Add entries for potentially generated headers. Also remove a couple of items already present in lib/.gitignore. * cfg.mk (sc_gitignore_missing): A new syntax check rule to identify missing .gitignore entries. (sc_gitignore_redundant): A new syntax check rule to identify redundant .gitignore entries. Reported by Tomas Nordin. 2015-06-06 Pádraig Brady tests: fix false failure in recent test adjustment * configure.ac: Comment on why we link rather than run the test, and remove the moot __ELF__ check since we never ran it anyway, and the new CFLAGS and LDFLAGS are a more direct test of support. * tests/misc/wc-parallel.sh: Fix a syntax error in the previous change. * tests/misc/md5sum-parallel.sh: Use better error checking, consistent with that used in wc-parallel.sh. 2015-06-06 Paul Eggert build: port to AIX Problems reported by Michael Felt, and and part of this fix taken from code suggested by Pádraig Brady in: http://bugs.gnu.org/20733#112 * configure.ac (stdbuf_supported): Check for warnings, and for -fPIC and -shared, for AIX. * src/stat.c (STRUCT_STATVFS): Define to struct statvfs64 if STATFS is statvfs64. * src/sync.c (sync_arg) [_AIX]: Open in write mode, since AIX fsync doesn't work on read-only file descriptors. * tests/misc/wc-parallel.sh: Skip test if xargs -P does not work. build: update gnulib submodule to latest 2015-06-05 Paul Eggert build: update gnulib submodule to latest 2015-06-04 Paul Eggert build: port single_binary_prog to POSIX shell Problem reported privately by Michael Felt. * Makefile.am (install-exec-hook): * src/local.mk (src/coreutils_symlinks, src/coreutils_shebangs) (clean-local): Port to POSIX shell, which doesn't allow 'for i in ; do ...'. 2015-06-03 Pádraig Brady build: add a dependency on the gnulib tempname module * bootstrap.conf: Add "tempname" which is needed by mktemp(1). The explicit dependency supports running gnulib-tool with the --conditional-dependencies option, used to minimize built modules. Note on a Fedora 22 system, that results in avoiding redundant builds of: areadlinkat.o asnprintf.o fd-hook.o fseterr.o printf-args.o printf-parse.o sockets.o vasnprintf.o. However --conditional-dependencies is not enabled, since it currently precludes the inclusion of gnulib tests. 2015-06-02 Pádraig Brady build: update to latest gnulib Mainly with build fixes for Mac OS X. 2015-06-01 Pádraig Brady tests: fix race in tail test without inotify * tests/tail-2/wait.sh: Without inotify, skip a portion of the test that checks that -F never outputs from a tailed descriptor after the followed name is recreated, because tail_forever() doesn't guarantee that. Noticed at http://hydra.nixos.org/build/22766288 tests: fix false failure on loaded systems * tests/misc/uniq-perf.sh: Use our standard 10s timeout, which is sufficient to trigger the failure and also avoids a false failure on slow/loaded systems. Noticed at http://hydra.nixos.org/build/22766288 2015-05-30 Pádraig Brady maint: avoid new coverity warnings * src/sync.c (sync_arg): Initialise variable to avoid unitialized access if assert is disabled. * src/head.c (elide_tail_bytes_file): Support this function with ---presume-input-pipe and larger files, which regressed with commit v8.23-47-g2662702. (elide_tail_lines_file): Likewise. * src/dd.c (dd_copy): Explicitly don't try to ftruncate() upon failure to lseek() (the existing check against st_size was already protecting that). * src/factor.c (factor_using_squfof): Assert (only when linting due to performance) to avoid the implication of divide by zero. * src/od.c (read_block): Remove dead code. * src/tac.c (tac_seekable): Likewise. * src/ls.c (gobble_file): Likewise. 2015-05-28 Pádraig Brady build: remove workarounds for unsupported gettext versions Now that we depend on gettext >= 0.19.2 remove the workaround for issues in autopoint 0.18.3. Note the scheme currently used in newer gettext (autopoint) to avoid these issues requires autoconf >= 2.69, therefore we update this requirement also. Note the gettext version dependence from gnulib comes from gnulib using gettext macros, and coreutils indirectly depends on the gettext module due to: http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=41dca647 bootstrap will then update m4/po.m4 and thus require a supportng gettext version. * bootstrap: Remove moot warning (resyncing with gnulib). * configure.ac (AC_PREREQ): Change to 2.69 (now 3 years old). 2015-05-28 Pádraig Brady tests: ln/hard-to-sym FAIL rather than ERROR when ln fails * tests/ln/hard-to-sym.sh: Only call framework_failure_ when ln returns success. copy: prefer our hardlink to symlink emulation on OS X 10.10 * src/copy.c (CAN_HARDLINK_SYMLINKS): Don't enable use of linkat() on Darwin 14, as the gnulib fallback emulation there doesn't preserve ownership and timestamps etc. This fixes a test failure in tests/cp/link-symlink.sh * tests/cp/link-deref.sh: Adjust accordingly. 2015-05-26 Pádraig Brady maint: remove sys/types.h include order check * src/system.h: This was inadvertently ineffective due to a typo in commit v8.9-10-ge1aaf89 (Jan 2011), but has not caused any issues, so remove. 2015-05-23 Pádraig Brady mkdir: fix -pZ with existing parent directories When the parent directory exists and has a different default context to the final directory, the context was incorrectly left as that of the parent directory. * src/mkdir.c (process_dir): Because defaultcon() is called for existing ancestors (as it must be to avoid races), then we must unconditionally call restorecon() on the last component due to the already documented caveat with make_dir_parents(). Alternatively you could temp disable o->set_security_context around make_dir_parents(), but that would be subject to races. * tests (tests/mkdir/restorecon.sh): Add a TODO for improvement. Reference mknod and mkfifo with print_ver_. * NEWS: Mention the bug fix. Fixes http://bugs.gnu.org/20616 2015-05-22 Assaf Gordon tests: df-output: accept multiple spaces in header * tests/df/df-output.sh: Allow for multiple spaces in the header line of 'df', resulting from alignment with disk sizes >= 10TB. 2015-05-19 Pádraig Brady tests: fix non POSIX constructs causing failures with dash * tests/cp/no-ctx.sh: Scope of `var=val func` is inconsistent across shells, so avoid that construct with functions. * tests/df/no-mtab-status.sh: Likewise. * tests/tail-2/inotify-race.sh: `read` needs an argument. * tests/tail-2/inotify-race2.sh: Likewise. 2015-05-17 Pádraig Brady build: fix 'dist' and 'syntax-check' targets in VPATH build * cfg.mk: Various syntax-check adjustments so that it's not assumed the $builddir is the base distribution directory. * Makefile.am: Likewise for the 'dist' target. build: avoid issues with case insensitive file systems * cfg.mk (sc_case_insensitive_file_names): A new syntax-check rule. * tests/tail-2/descriptor-vs-rename.sh: Rename from tests/tail-2/f-vs-rename.sh * tests/local.mk: Reference the renamed test. Reported by Jim Meyering. 2015-05-14 Bernhard Voelker tests: fix check for local file system in inotify-rotate-resources.sh * tests/tail-2/inotify-rotate-resources.sh: s/(is_local_dir)/\1_/ 2015-05-14 Pádraig Brady tests: fix async allocation race on BTRFS * tests/dd/sparse.sh: Sync files before checking allocations, which may be done asynchronously on NFS and BTRFS at least. Also mark this test as very expensive on remote file systems. * tests/du/2g.sh: Likewise, also use fallocate if available to efficiently allocate the large file, otherwise skip on remote file systems. * tests/tail-2/inotify-rotate-resources.sh: Use the more standard is_local_dir_() to check remoteness. * tests/cp/fiemap-empty.sh: Comment on the sync issue for this currerntly unused test. Fixes http://bugs.gnu.org/20570 2015-05-13 Pádraig Brady timeout: with --foreground don't send SIGCONT * src/timeout.c (cleanup): Don't send SIGCONT to the monitored program when --foreground is specified, as it's generally not needed for foreground programs, and can cause intermittent signal delivery issues with monitors like GDB for example. * doc/coreutils.texi (timeout invocation): Mention that SIGCONT is not sent with --foreground. * NEWS: Mention the behavior change. 2015-05-13 Pádraig Brady split: auto set suffix len for --numeric-suffixes= doc: clarify the operation of wc -L * src/wc.c (usage): State that it calculates display width. * doc/coreutils.texi (wc invocation): Detail the distinct items used to determine the display width. 2015-05-11 Pádraig Brady tail: consistently output all data for truncated files Generally if logs are truncated, they're truncated to 0 length, so output all existing data when our heuristic determines truncation. Note with inotify, truncate() and write() are often determined independently and so all data would be written if that was the case. * src/tail.c (check_fspec): Reset file offset to 0 upon truncation. (tail_forever): Likewise. (recheck): Add a FIXME for the related issue where tail may lose data due to tail discounting older log files too early. * tests/tail-2/truncate.sh: A new test. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. 2015-05-11 Pádraig Brady tail: fix inotify startup races The previous fixes to races in the various tail tests, identified actual races in the tail inotify implementation. With --follow=descriptor, if the tailed file was replaced before the inotify watch was added, then any subsequent changes were ignored. Similarly in --follow=name mode, all changes to a new name were effectively ignored if that name was created after the original open() but before the inotify_add_watch(). * src/tail.c (tail_forever_inotify): Fix 3 cases. 1. With -f, don't stop tailing when file removed before watch. 2. With -f, watch right file when file replaced before watch. 3. With -F, inspect correct file when replaced before watch. Existing tests identify these when tail compiled with TAIL_TEST_SLEEP. * tests/tail-2/inotify-rotate-resources.sh: This test also identifies the issue with --follow=name when TAIL_TEST_SLEEP is used. Adjust so the test is immune to such races, and also fail quicker on remote file systems. * tests/tail-2/inotify-race2.sh: A new test using GDB, based on inotify-race.sh, which tests the -F race without needed recompilation with sleeps. * tests/local.mk: Reference the new test. * NEWS: Mention the bug. 2015-05-11 Pádraig Brady tests: cleanup background processes upon interruption Reap background processes so that: - Stray processes aren't left on the system - Files aren't held open causing deletion issues on NFS - Partitions used to run the tests from can be unmounted * tests/tail-2/F-vs-missing.sh: Add the `kill && wait` of the background $pid(s) to cleanup_(). * tests/tail-2/F-vs-rename.sh: Likewise. * tests/tail-2/f-vs-rename.sh: Likewise. * tests/tail-2/append-only.sh: Likewise. * tests/tail-2/assert-2.sh: Likewise. * tests/tail-2/assert.sh: Likewise. * tests/tail-2/flush-initial.sh: Likewise. * tests/tail-2/inotify-hash-abuse.sh: Likewise. * tests/tail-2/inotify-hash-abuse2.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: Likewise. * tests/tail-2/inotify-rotate.sh: Likewise. * tests/tail-2/pid.sh: Likewise. * tests/tail-2/pipe-f2.sh: Likewise. * tests/tail-2/retry.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/tail-n0f.sh: Likewise. * tests/tail-2/wait.sh: Likewise. * tests/cp/existing-perm-race.sh: Likewise. * tests/cp/file-perm-race.sh: Likewise. * tests/cp/parent-perm-race.sh: Likewise. * tests/cp/sparse-to-pipe.sh: Likewise. * tests/dd/stats.sh: Likewise. * tests/du/move-dir-while-traversing.sh: Likewise. * tests/misc/cat-buf.sh: Likewise. * tests/misc/help-version.sh: Likewise. * tests/misc/printf-surprise.sh: Likewise. * tests/misc/sort-compress-proc.sh: Likewise. * tests/misc/sort-spinlock-abuse.sh: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/misc/tac-continue.sh: Likewise. * tests/misc/timeout-group.sh: Likewise. * tests/mv/i-3.sh: Likewise. * tests/rm/dangling-symlink.sh: Likewise. * tests/rm/isatty.sh: Likewise. * cfg.mk (sc_prohibit_test_background_without_cleanup_): A new syntax-check to ensure cleanup_() is defined when background tasks are created in a test. 2015-05-11 Pádraig Brady tests: avoid hung processes due to gdb SIGCONT handling * tests/tail-2/inotify-race.sh: Add a `wait` to ensure that we reap all background gdb and tail processes. That resulted in the test hanging intermittently and upon investigation was due to gdb intermittently failing to terminate the child process due to receiving a SIGCONT signal. Therefore we avoid using timeout(1) which sends that signal, and instead rely on tail's inbuilt --pid monitoring on a background sleep process. Given this new implementation, the VERY_EXPENSIVE guard was removed. Related issues with this test hanging were previously discussed at: https://lists.gnu.org/archive/html/bug-coreutils/2009-12/msg00025.html tests: fix races in and standardize the tail tests * tests/tail-2/F-vs-missing.sh: Use standard "fastpoll" options (-s.1 --max-unchanged-stats=1) to speedup the non-inotify case. Add the non-inotify case to the test. `wait` on the background tail process to terminate which should avoid the need for the non standard `retry_delay_ cleanup ...` on NFS. * tests/tail-2/F-vs-rename.sh: Remove 'out' at the start of the loop, to avoid a race in checking its contents. Also ensure 'a' & 'b' files are present before the tail process starts. Use the standard "fastpoll" options as above. * tests/tail-2/f-vs-rename.sh: Likewise. * tests/tail-2/append-only.sh: Use more standard variable names. * tests/tail-2/flush-initial.sh: Use "fastpoll" options for non-inotify platforms. Also `wait` on the background tail to avoid stray processes and file cleanup issues on NFS. * tests/tail-2/inotify-hash-abuse.sh: Always run non-inotify case. Use "fastpoll" options. Use a more standard retry_delay_ instead of a hardcoded sleep loop. Add a `wait` on the background tail. * tests/tail-2/inotify-hash-abuse2.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: Wait just on the specific tail $pid needed. * tests/tail-2/inotify-rotate.sh: Use "fastpoll" options. * tests/tail-2/pid.sh: Use standard variable names. Add a `wait` on the background tails. * tests/tail-2/pipe-f2.sh: Likewise. * tests/tail-2/tail-n0f.sh: Likewise. * tests/tail-2/retry.sh: Use "fastpoll" options. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/wait.sh: Likewise. Speedup by using sub second parameters to timeout(1). Improve the part ensuring that -F never follows a renamed file. * tests/tail-2/infloop-1.sh: Remove invalid test. tail(1) was not being passed the --pid=$yes_pid option, retry_delay_ wasn't used to avoid races, and yes could write huge files before being killed. * tests/local.mk: Remove the invalid test reference. * tests/tail-2/assert-2.sh: Rewrite using retry_delay_(). Since no longer hardcoding large delays, remove the VERY_EXPENSIVE tag. * tests/tail-2/assert.sh: Likewise. 2015-05-10 Paul Eggert build: rewrite is_ENOTSUP without an #if directive * src/system.h (is_ENOTSUP): Avoid in-function #if directive. 2015-05-10 Jim Meyering build: avoid a warning form gcc's new -Wlogical-op Without this change, very recent gcc (e.g., version 6.0.0 20150509) would print the following when configured with --enable-gcc-warnings: src/copy.c:165:30: error: logical 'or' of equal expressions \ [-Werror=logical-op] && (errno == EOPNOTSUPP || errno == ENOTSUP || errno == ENOSYS)) ^ * src/system.h (is_ENOTSUP): New function. * src/copy.c (punch_hole): Use it. * src/ls.c (errno_unsupported, gobble_file): Use it. 2015-04-30 Pádraig Brady doc: standardize messages about the '-' stdin FILE * src/system.h (emit_stdin_note): A new function, refactoring the usage note about the '-' FILE implying stdin. * src/base64.c (usage): Use the new function to emit the note in a standard location and with standard separation. * src/cat.c (usage): Likewise. * src/csplit.c (usage): Likewise. * src/cut.c (usage): Likewise. * src/expand.c (usage): Likewise. * src/fmt.c (usage): Likewise. * src/head.c (usage): Likewise. * src/md5sum.c (usage): Likewise. * src/nl.c (usage): Likewise. * src/od.c (usage): Likewise. * src/paste.c (usage): Likewise. * src/pr.c (usage): Likewise. * src/ptx.c (usage): Likewise. * src/shred.c (usage): Likewise. * src/shuf.c (usage): Likewise. * src/sort.c (usage): Likewise. * src/sum.c (usage): Likewise. * src/tac.c (usage): Likewise. * src/tail.c (usage): Likewise. * src/tsort.c (usage): Likewise. * src/unexpand.c (usage): Likewise. * src/wc.c (usage): Likewise. * src/join.c (usage): Adjust the separation used for the message referring to FILE1 or FILE2 as stdin. * src/comm.c (usage): Add a message using the same wording (translation) as used in join. * src/split.c (usage): Reword to using FILE rather than INPUT, allowing use of emit_stdin_note(). Also remove the mention of "fixed-size" pieces as this isn't now always the case. Fixes http://pad.lv/1450179 tests: don't skip df tests with /proc/self/mountinfo * tests/df/no-mtab-status.sh: getmntent is no longer called when /proc/self/mountinfo is present, thus causing the test to be skipped. Therefore wrap fopen() to ignore mountinfo, and use the test genmntent table instead. * tests/df/skip-duplicates.sh: Likewise. 2015-04-28 Yunlian Jiang build: fix SINGLE_BINARY build when printf is a macro * src/coreutils.c (usage): include coreutils.h outside the printf call, because if it's a macro you will get the error: embedding a #include directive within macro arguments is not supported 2015-04-28 Pádraig Brady maint: avoid a new -Werror=return-type warning in yes.c * src/yes.c (main): Simplify the logic so that the compiler can see this function always returns a value. This was seen with GCC 5.0 in SINGLE_BINARY mode. tests: run ls capability coloring test irrespective of $LS_COLORS * tests/ls/no-cap.sh: Ensure the test isn't skipped even if capability coloring is disabled in the current $LS_COLORS. Also just enable/disable capability coloring to avoid the dircolors(1) overhead. 2015-04-27 Pádraig Brady maint: use gnulib styling with the online manual The equivalent styling added in v8.23-155-g3b98ee7, is now applied to gnulib using projects by default. build: rely on gnulib to determine printf routines are safe gnulib now only checks that the printf routines never crash, which is all coreutils currrently requires, and so we revert commit v8.23-81-gf57bfbb to let gnulib decide whether to replace the system printf routines. maint: fix printf format for signed integers With GCC 5 and the newly added warnings from gnulib, ensure the correct signed integer is passed for the printf format, to avoid -Werror=format= failures. build: update gnulib submodule to latest Fix file-has-acl build failure on RHEL/Centos 6. Fix GCC 5 warnings with printf and in fts.c. build: reduce gettext dependency to 0.19.2 * bootstrap.conf: 0.19.2 is available on openSUSE-13.2, Debian-8.0, and Ubuntu-14.10. Given there were issues with earlier 0.19 gettext releases, set this as the new minimum. * configure.ac (AM_GNU_GETTEXT_VERSION): Likewise. Reported by Bernhard Voelker 2015-04-27 Paul Eggert ls: on GNU/Linux, remove dependency on libacl * src/local.mk (src_ls_LDADD): Change from LIB_ACL to LIB_HAS_ACL. build: update gnulib submodule to latest * bootstrap.conf (gnulib_modules): Add file-has-acl. (buildreq): Bump autopoint and gettext to 0.19.4. * configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.19.4. * gl/lib/tempname.c.diff, gl/lib/tempname.h.diff: Merge recent gnulib changes. 2015-04-23 Michael Witten doc: fix grammar issue in truncate info * doc/coreutils.texi (truncate invocation): The word 'their' is incorrect; 'each file' is the antecedent, and is singular, so 'its' is the correct pronoun. 2015-04-22 Pádraig Brady build: fix potential factor build failure on arm and mips * src/longlong.h: Sync with the latest longlong.h from libgmp to: - Use __builtin_c[lt]zl on arm64. - Fix sparc64 vis3 build failure due to missing __clz_tab. - Avoid a clang build issue on mips. - Support thumb2 arm 32 bit system. * src/cfg.mk (sc_ensure_comma_after_id_est): Exclude longlong.h to ease merges. 2015-04-22 Pádraig Brady maint: avoid -Werror=strict-overflow warnings with GCC 5 All warnings were of the form: "assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow]" * src/dd.c (cache_round): Use an appropriately sized unsigned type, to avoid possibility of undefined signed overflow. * src/mknod.c (main): Likewise. * src/pr.c (pad_down): Likewise. * src/wc.c (main): Likewise. * src/tail.c (main): Assert that argc >= 0 thus allowing the compiler to assume without implication that argc - optind is positive. 2015-04-17 Pádraig Brady dircolors: add 'MISSING' to the default database * src/dircolors.hin: Add the MISSING entry, to indicate this as a possibility in new templates output from dircolors, and also to ease comparison with existing databases that generally do define a MISSING entry. 2015-04-13 Andreas Gruenbacher doc: clarify how cp behaves with default ACLs * doc/coreutils.texi (cp invocation): Mention that when copying files without preserving permissions, the umask or a default ACL affect the mode of new files. * THANKS.in: Remove committer. Related to http://bugs.gnu.org/8527 2015-04-13 Pádraig Brady df: fix --local hanging with inaccessible remote mounts * src/df.c (filter_mount_list): With -l, avoid stating remote mounts. * init.cfg: Avoid test hangs with inaccessible remote mounts. * tests/df/no-mtab-status.sh: Skip with inaccessible remote mounts. * tests/df/skip-rootfs.sh: Likewise. * tests/df/total-verify.sh: Likewise. * NEWS: Mention the bug fix. Reported at http://bugzilla.redhat.com/1199679 2015-04-09 Mitchel Humpherys doc: correct pluralization for mkfifo and mknod * doc/coreutils.texi: `mkfifo' and `mknod' use the optContext macro which adds a description for the SELinux security context in addition to the single option already described in each case. The result in both cases is two options being introduced as `option' (singular). Fix this by introducing them as `options' (plural). 2015-04-03 Pádraig Brady df: fix use of uninitialized variable reported by valgrind Conditional jump or move depends on uninitialised value(s) at 0x40380C: get_field_values (df.c:840) by 0x403E16: get_dev (df.c:994) by 0x404D65: get_all_entries (df.c:1364) by 0x405926: main (df.c:1714) * src/df.c (get_dev): Initialize the fsu.fsu_bavail_top_bit_set member, when adding placeholder entries. (main): Avoid a "definitely lost" memory leak warning from valgrind, reported by Bernhard Voelker. 2015-04-01 Pádraig Brady doc: clarify that ls --sort=time is newest first * src/ls.c (usage): Add punctuation to avoid ambiguity in the description of the --time option. Mention that both the -u and --sort=time default order is newest first. 2015-03-31 Stephane Chazelas tail: fix -f to follow changes after a rename * src/tail.c (tail_forever_inotify): Only monitor write()s and truncate()s to files in --follow=descriptor mode, thus avoiding the bug where we removed the watch on renamed files. Also adjust the inotify event processing code that is now significant only in --follow=name mode. * tests/tail-2/F-vs-rename.sh: Improve this existing test by running in both polling and inotify modes. * tests/tail-2/f-vs-rename.sh: A new test based on the existing one. * tests/local.mk: Reference the new test. * NEWS: Mention the bug. Fixes http://bugs.gnu.org/19760 2015-03-29 Pádraig Brady doc: move numfmt info to the 'Numeric operations' section * doc/coreutils.texi: Move numfmt info to this section, as numfmt functionality aligns more with seq and factor, than fmt and pr etc. 2015-03-27 Paul Eggert nohup: clarify stdin redirection Problem reported by Isaac Schwabacher in: http://bugs.gnu.org/20214 * doc/coreutils.texi (nohup invocation): Clarify that when nohup's stdin gets redirected, it's unreadable. * doc/coreutils.texi (nohup invocation): * src/nohup.c (usage): Don't promise /dev/null. 2015-03-27 Assaf Gordon tests: fix false test failure with df on Debian/kFreeBSD * tests/fs/skip-duplicates.sh: On this platform .mnt_opts is significant so define to empty to avoid a NULL deref in read_file_system_list(). Fixes http://bugs.gnu.org/20210 2015-03-26 Christoph Anton Mitterer doc: disambiguate the ls --color description * src/ls.c (usage): Avoid the implication that the default ls behavior is to --color=always. Reported in http://bugs.debian.org/781208 2015-03-26 Pádraig Brady doc: clarify the date standard output formats * src/date.c (usage): Use FMT rather than TIMESPEC as the parameter, since it's simpler to understand and can be better aligned. Give an example for the --iso-8601 output format. Adjust the example used for the 3 standard formats to be unambiguous with respect to day/mon ordering and use of leading zeros in the time. Reorder the options descriptions slightly, so that the 3 standards options are together. Indent the multi-line descriptions so that grouping is obvious. Remove a redundant description of the --rfc-3339 format, which is obvious in the existing example. Separate these 3 standards options to their own translatable string to simplify translation. Change 'date and time' to 'date/time' in the --iso-8601 description to be consistent with --rfc-3339 and to help avoid the implication that the time is always output or even output by default. Fixes http://bugs.gnu.org/20203 2015-03-24 Pádraig Brady maint: apply basic styling to online manual * README-release: Reference http://www.gnu.org/s/coreutils/manual.css to apply basic styling to the online coreutils manual, consistent with the Emacs documentation. doc: clarify the uniq -D man page description * src/uniq.c (usage): The description was very confusing in the man page due to the stripped newlines. Add punctuation for clarification. wc: use a more adaptive wc -l implementation * src/wc.c (wc): Allow any block to select the count implementation, rather than just using the first 10 lines. This also simplifies the code from 3 loops to 2. 2015-03-23 Dan Jacobson doc: clarify default order for ls --sort=size * src/ls.c (usage): Mention that default order is largest first. Fixes http://bugs.gnu.org/20172 2015-03-20 Pádraig Brady maint: really fix wildcard quoting in sc_tests_executable * cfg.mk (sc_tests_executable): The previous commit avoided the globbing, but also passed on the quoted wildcards to find(1). We could use eval to handle the quoting, though that's a bit awkward and dangerous, so instead explicitly disable globbing for the whole make target subshell. Note noglob is not available on solaris, where we fall back to set -f. Note also that zsh uses set -F for this, but that's moot here. Also correct the find(1) expression to include the -o between each wildcard. maint: fix wildcard quoting in sc_tests_executable * cfg.mk (sc_tests_executable): If there are files with $TEST_EXTENSIONS in the current directory, then the lack of quoting of the $test_extensions_rx contents could result in globbing and an inconsequential run. find(1) produces warnings only with more than one expansion. 2015-03-20 Kristoffer Brånemyr wc: speedup counting of short lines Using a test file generated with: yes | head -n100M > 2x100M.txt before> time wc -l 2x100M.txt real 0.842s user 0.810s sys 0.033s after> time wc -l 2x100M.txt real 0.142s user 0.111s sys 0.031s * src/wc.c (wc): Split the loop that deals with -l into 3. The first is used at the start of the input to determine if the average line length is < 15, and if so the second loop is used to look for '\n' internally to wc. For longer lines, memchr is used as before to take advantage of system specific optimizations which any outweigh function call overhead. Note the first 2 loops could be combined, though in testing, GCC 4.9.2 at least, wasn't sophisticated enough to separate the loops based on the "check_len" invariant. Note also __builtin_memchr() isn't significant here as GCC currently only applies constant folding with that. * NEWS: Mention the improvement. 2015-03-10 Giuseppe Scrivano yes: improve efficiency when all args aren't buffered * src/yes.c (main): Even when the internal buffer isn't large enough, output what we've buffered already, and interate over the rest. This improves the performance in the edge case where there are many small arguments that overflow the buffer. * tests/misc/yes.sh: Add a test case for the many small arguments case. 2015-03-10 Pádraig Brady yes: output data more efficiently yes(1) may be used to generate repeating patterns of text for test inputs etc., so adjust to be more efficient. Profiling the case where yes(1) is outputting small items through stdio (which was the default case), shows the overhead of continuously processing small items in main() and in stdio: $ yes >/dev/null & perf top -p $! 31.02% yes [.] main 27.36% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5 14.51% libc-2.20.so [.] fputs_unlocked 13.50% libc-2.20.so [.] strlen 10.66% libc-2.20.so [.] __GI___mempcpy 1.98% yes [.] fputs_unlocked@plta Sending more data per stdio call improves the situation, but still, there is significant stdio overhead due to memory copies, and the repeated string length checking: $ yes "`echo {1..1000}`" >/dev/null & perf top -p $! 42.26% libc-2.20.so [.] __GI___mempcpy 17.38% libc-2.20.so [.] strlen 5.21% [kernel] [k] __srcu_read_lock 4.58% [kernel] [k] __srcu_read_unlock 4.27% libc-2.20.so [.] _IO_file_xsputn@@GLIBC_2.2.5 2.50% libc-2.20.so [.] __GI___libc_write 2.45% [kernel] [k] system_call 2.40% [kernel] [k] system_call_after_swapgs 2.27% [kernel] [k] vfs_write 2.09% libc-2.20.so [.] _IO_do_write@@GLIBC_2.2.5 2.01% [kernel] [k] fsnotify 1.95% libc-2.20.so [.] _IO_file_write@@GLIBC_2.2.5 1.44% yes [.] main We can avoid all stdio overhead by building up the buffer _once_ and outputting that, and the profile below shows the bottleneck moved to the kernel: $ src/yes >/dev/null & perf top -p $! 15.42% [kernel] [k] __srcu_read_lock 12.98% [kernel] [k] __srcu_read_unlock 9.41% libc-2.20.so [.] __GI___libc_write 9.11% [kernel] [k] vfs_write 8.35% [kernel] [k] fsnotify 8.02% [kernel] [k] system_call 5.84% [kernel] [k] system_call_after_swapgs 4.54% [kernel] [k] __fget_light 3.98% [kernel] [k] sys_write 3.65% [kernel] [k] selinux_file_permission 3.44% [kernel] [k] rw_verify_area 2.94% [kernel] [k] __fsnotify_parent 2.76% [kernel] [k] security_file_permission 2.39% yes [.] main 2.17% [kernel] [k] __fdget_pos 2.13% [kernel] [k] sysret_check 0.81% [kernel] [k] write_null 0.36% yes [.] write@plt Note this change also ensures that yes(1) will only write complete lines for lines shorter than BUFSIZ. * src/yes.c (main): Build up a BUFSIZ buffer of lines, and output that, rather than having stdio process each item. * tests/misc/yes.sh: Add a new test for various buffer sizes. * tests/local.mk: Reference the new test. Fixes http://bugs.gnu.org/20029 2015-03-07 Pádraig Brady build: fix make dependencies for test.1 In certain parallel build situations this would give the error: help2man: can't get `--help' info from man/test.td/[ Makefile:14189: recipe for target 'man/test.1' failed * man/local.mk (test.1): Depend on `[` rather than `test`, as `test --help` outputs nothing. Also move dir.1 and vdir.1 back to the main list, as they're no more exceptions than sha1sum etc. 2015-03-05 Bernhard Voelker maint: be less strict about executable permissions of tests With "umask 0027" or even "umask 0077", the git clone of coreutils does not have the executable bit set for 'other' (or 'group). Therefore, "make syntax-check" would fail. * cfg.mk (sc_tests_executable): Change the -perm argument of find(1) to only print the names of the files which are not executable by the user, rather than insisting on ugo+x (octal 111). 2015-03-05 Stéphane Aulery doc: mention persistence caveats in sync man page * man/sync.x ([BUGS]): Mention the caveats and reference the system call man pages for more details. Fixes http://bugs.gnu.org/19995 Debian bug #507085 reported by jidanni@jidanni.org 2015-03-04 Pádraig Brady tee: generalize the --write-error option to --output-error Adjust commit v8.23-140-gfdd6ebf to add the --output-error option instead of --write-error, and treat open() errors like write() errors. * doc/coreutils.texi (tee invocation): s/write-error/output-error/. * src/tee.c (main): Exit on open() error if appropriate. * tests/misc/tee.sh: Add a case to test open() errors. * NEWS: Adjust for the more general output error behavior. Suggested by Bernhard Voelker. 2015-03-04 Jarosław Gruca maint: update stale comment about ls color sequences * src/ls.c (color_indicator[C_END]): Comment with the correct sequence, which was used since commit v6.10-61-g483297d Fixes http://bugs.gnu.org/19992 2015-02-27 Shane M Seymour tail,stat: improve support for the IBRIX file system Note that IBRIX used to have a different magic number 0x013111A7 instead of the current 0x013111A8. However, the former is no longer used and the version of IBRIX it was used in is really ancient, so it's extremely unlikely anyone is still using it. Therefore, just add the newer magic number. Mark IBRIX as a 'remote' file system type as inotify support had never been officially tested with it. * src/stat.c (human_fstype): Add file system ID definition. * NEWS: Mention the improvement. Fixes http://bugs.gnu.org/19951 2015-02-24 Pádraig Brady tee: add --write-error to control handling of closed pipes tee is very often used with pipes and this gives better control when writing to them. There are 3 classes of file descriptors that tee can write to: files(1), pipes(2), and early close pipes(3). Handling write errors to 1 & 2 is supported at present with the caveat that failure writing to any pipe will terminate tee immediately. Handling write errors to type 3 is not currently supported. To improve the supported combinations we add these options: --write-error=warn Warn if error writing any output including pipes. Allows continued writing to still open files/pipes. Exit status is failure if any output had error. --write-error=warn-nopipe, -p Warn if error writing any output except pipes. Allows continued writing to still open files/pipes. Exit status is failure if any non pipe output had error. --write-error=exit Exit if error writing any output including pipes. --write-error=exit-nopipe Exit if error writing any output except pipes. Use the "nopipe" variants when files are of types 1 and 3, otherwise use the standard variants with types 1 and 2. A caveat with the above scheme is that a combination of pipe types (2 & 3) is not supported robustly. I.e. if you use the "nopipe" variants when using both type 2 and 3 pipes, then any "real" errors on type 2 pipes will not be diagnosed. Note also a general issue with type 3 pipes that are not on tee's stdout, is that shell constructs don't allow to distinguish early close from real failures. For example `tee >(head -n1) | grep -m1 ..` can't distinguish between an error or an early close in "head" pipe, while the fail on the grep part of the pipe is distinguished independently from the resulting pipe errors. This is a general issue with the >() construct, rather than with tee itself. * NEWS: Mention the new feature. * doc/coreutils.texi (tee invocation): Describe the new option. * src/tee.c (usage): Likewise. (main): With --write-error ignore SIGPIPE, and handle the various exit, diagnostics combinations. * tests/misc/tee.sh: Tess all the new options. Fixes http://bugs.gnu.org/11540 2015-02-23 Bernhard Voelker tee: close "-" file This is a cleanup to the previous commit v8.23-138-g7ceaf1d. * src/tee.c (tee_files): Do not exempt the "-" file from being closed, as this is no longer stdout but a normal file. 2015-02-20 Bernhard Voelker tee: treat '-' operand as file name as mandated by POSIX Since v5.2.1-1247-g8dafbe5, tee(1) treated '-' as stdout while POSIX explicitly requires to treat this as a file name. Revert this change, as the interleaved output - due to sending another copy of input to stdout - is not considered to be useful. Discussed in http://lists.gnu.org/archive/html/coreutils/2015-02/msg00085.html * src/tee.c (tee_files): Remove the special handling for "-" operands. (usage): Remove the corresponding sentence. * doc/coreutils.texi (common options): Remove the "tee -" example. (tee invocation): Document that tee(1) now treats "-" as a file name. * tests/misc/tee.sh: Add a test case for "tee -". While at it, re-indent the above multi-argument processing case and extend that to 13 operands, as POSIX mandates that, too. * tests/misc/tee-dash.sh: Remove now-obsolete test. * tests/local.mk (all_tests): Remove the above test. * NEWS (Changes in behavior): Mention the change. 2015-02-18 Pádraig Brady tee: exit early if no more writable outputs * src/tee.c (main): Don't continue reading if we can't output anywhere. * tests/misc/tee.sh: Ensure we exit when no more outputs. * NEWS: Mention the change in behavior. tests: support stderr verification with returns_() * tests/init.sh (returns_): Disable tracing for this wrapper function, so that stderr of the wrapped command is unchanged, allowing for verification of the contents. 2015-02-17 Bernhard Voelker maint: prefer STREQ_LEN and STRPREFIX over strncmp in all cases * cfg.mk (sc_prohibit_strncmp): Improve the search pattern: use _sc_search_regexp to find all invocations of strncmp except when used on a macro definition line; just match the function name with an opening parenthesis. Before, the expression missed places where the comparison against 0 was in a subsequent line. * src/system.h (STRNCMP_LIT): Shorten 'literal' to 'lit' to move the whole definition of the macro into one line - thus making sc_prohibit_strncmp pass. (STRPREFIX): Add space before parenthesis. * src/du.c (main): Prefer STREQ_LEN over strncmp. * src/pinky.c (scan_entries): Likewise. * src/tac.c (tac_seekable): Likewise. * src/who.c (scan_entries): Likewise. 2015-02-17 Pádraig Brady stty: fix setting of 'extproc' on BSD This setting is unusual on BSD as it's read normally in the local flags returned by tcgetattr(), but can only be set with an ioctl. Setting with tcsetattr() is ignored. * src/stty.c (NO_SETATTR): A new flag to indicate the setting is read and displayed like a normal termios flag, but is set in some other manner. (main): Skip tcsetattr() for this setting when this flag is set. Also fixup the exiting 'extproc' processing to handle the '-extproc' case correctly. (sane_mode): Skip setting '-extproc' for 'sane' to avoid the error. This isn't ideal but matches the operation of the BSD native stty. 2015-02-12 Bernhard Voelker maint: enhance '.mailmap' mappings for THANKS generation * .mailmap (jeff.liu@oracle.com): There are 3 different names in the 'git log' output for this email address; choose "Jeff Liu" as canonical form. (Алексей Шилин): Convert name to latin1 ("Aleksej Shilin") to improve the sort order of the generated 'THANKS' file. 2015-02-12 Bernhard Voelker maint: sort THANKS using our own gear At least 'sort' on openSUSE/Fedora have a bug in the case-folding code of their I18N downstream patch which leads to wrong sort results, e.g. "Dániel" coming after "Dylan". * Makefile.am (THANKS): Sort the final contributor list using our own sort implementation - as others may result in a different order; add a FIXME comment to remove this again once common platforms have a functional 'sort -f'. Add '-k1,1' for a better sort order. While at it, save a grep and perl call to prepare the list from 'THANKS.in' by doing all in the first perl call. 2015-02-11 Pádraig Brady maint: remove redundant diff filtering from sc_long_lines * cfg.mk (sc_long_lines): diff files are now completely excluded, so no need to filter portions of them. Reported by Bernhard Voelker. 2015-02-11 Pádraig Brady build: update to latest gnulib This includes a change to require --with-libmount to be used with configure, due to the many libmount dependencies. * bootstrap: Sync with gnulib to exit early on gnulib-tool error. * gl/lib/tempname.c.diff: Adjust for gnulib changes. * gl/lib/tempname.h.diff: Likewise. * gl/modules/tempname: Likewise. * doc/.gitignore: Add new gendocs_template_min gnulib script. 2015-02-11 Pádraig Brady maint: exclude diff files from long lines check * cfg.mk: Add .diff files to the exclusion expression for sc_long_lines, since the gnulib code might be >= 80 chars. Note 80 char lines trigger due to the added +/- diff marks. Also normalize the $$ used in the other sc_long_lines exclusion expressions. maint: avoid arbitrary memory access with buggy localtime() * src/ls.c (align_nstrftime): Be defensive and validate the tm_mon index before using to access the abmon array. This was _not_ seen to be an issue any system. See https://bugzilla.redhat.com/1190454 doc: give an example of using a seed for random operations * doc/coreutils.texi (Random sources): Give an example using openssl, generating a reproducible arbitrary amount of randomly distributed data, given a seed value. 2015-02-09 Bernhard Voelker maint: make sc_dd_O_FLAGS more robust The construct "diff ... || diff=1 || diff=" does not set the variable in all cases. This could be triggered with: $ env make diff=1 make sc_dd_O_FLAGS dd_O_FLAGS maint.mk: ./src/dd.c has inconsistent O_ flag lists cfg.mk:59: recipe for target 'sc_dd_O_FLAGS' failed make: *** [sc_dd_O_FLAGS] Error 1 * cfg.mk (sc_dd_O_FLAGS): Remember $? of the diff command directly and check its value later rather than using the above mentioned mapping. 2015-02-09 Bernhard Voelker maint: make sc_THANKS_in_duplicates more robust The previous commit v8.23-124-g7b1ca5f made the above syntax-check rule fail, because that took the whole content of THANKS.in for comparison. * cfg.mk (sc_THANKS_in_duplicates): Strip off the header (all before the first empty line) and the footer (all past the next empty line) from 'THANKS.in' for the check. 2015-02-09 Bernhard Voelker maint: sort contributors in THANKS.in * THANKS.in: Document the preferred sort order as a comment at the top of the file. Change "Марк Коренберг" to latin1 ("Mark Korenberg"). Sort all entries. * cfg.mk (sc_THANKS_in_sorted): Add rule to ensure that 'THANKS.in' remains sorted. 2015-02-06 Pádraig Brady maint: document a caveat in the tail inotify implementation * src/tail.c (main): Document another caveat with the inotify implementation wrt multiple hardlinked files. 2015-02-06 Pádraig Brady tail: return inotify resources where possible Each user has a maximum number of inotify watches, so handle the cases where we exhaust these resources. * src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch() the watch for an inode, when replacing with a new watch for a name. Return all used inotify resources when reverting to polling. Revert to polling upon first indication of inotify resource exhaustion. Revert to polling on any inotify resource exhaustion. Diagnose resource exhaustion correctly in all cases. Avoid redundant reinsertion in the hash for unchanged watches (where only attributes of the file are changed). * tests/tail-2/retry.sh: Avoid false failure when reverting to polling. * tests/tail-2/inotify-rotate.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. * tests/tail-2/inotify-rotate-resources.sh: New test to check that we're calling inotify_rm_watch() for replaced files. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. * THANKS.in: Thanks for reporting and problem identification. 2015-02-06 Pádraig Brady tests: fix recent regression in tail inotify test * tests/tail-2/inotify-rotate.sh (cleanup_fail_): Set fail=1 so that failures are identified. Regression in v8.23-63-g111a2b9 Also use print_ver_ rather than open coding --verbose support. Also check for more than a single 'b' which seems brittle. 2015-02-05 Yury Usishchev build: ensure make-prime-list doesn't access out of bounds memory The -fsanitize=address run associated with v8.22-75-gf940fec failed to check make-prime-list, as src/primes.h is not regenerated with `make clean`. Running with -fsanitize=address indicates a read 1 byte beyond the allocated buffer. $ rm src/make-prime-list.o $ make AM_CFLAGS=-fsanitize=address src/make-prime-list $ src/make-prime-list 5000 ================================================================= ==13913==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61e00000fa43 at pc 0x4016f5 bp 0x7fff9d9840e0 sp 0x7fff9d9840d0 READ of size 1 at 0x61e00000fa43 thread T0 #0 0x4016f4 in main src/make-prime-list.c:214 #1 0x7f98892c5fdf in __libc_start_main (/lib64/libc.so.6+0x1ffdf) #2 0x401774 (src/make-prime-list+0x401774) 0x61e00000fa43 is located 0 bytes to the right of 2499-byte region [0x61e00000f080,0x61e00000fa43) allocated by thread T0 here: #0 0x7f98896ba7b7 in malloc (/lib64/libasan.so.1+0x577b7) #1 0x400f3f in xalloc src/make-prime-list.c:163 #2 0x400f3f in main src/make-prime-list.c:198 SUMMARY: AddressSanitizer: heap-buffer-overflow src/make-prime-list.c:214 main Shadow bytes around the buggy address: 0x0c3c7fff9ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c3c7fff9f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c3c7fff9f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c3c7fff9f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c3c7fff9f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x0c3c7fff9f40: 00 00 00 00 00 00 00 00[03]fa fa fa fa fa fa fa 0x0c3c7fff9f50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fff9f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fff9f70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fff9f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c3c7fff9f90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa ... ==13913==ABORTING * src/make-prime-list.c (main): Bounds check the incremented index, before using to access the buffer. Fixes http://bugs.gnu.org/19784 2015-02-03 Pádraig Brady build: fix invalid gnulib patch * gl/lib/tempname.c.diff: Fix recent breakage so it applies again. Invalid patch was noticed at http://hydra.nixos.org/eval/1172233 * cfg.mk: Exempt diff files from these "id_est" syntax checks. (sc_ensure_gl_diffs_apply): A new syntax check, to ensure all patches under gl/ apply cleanly. Note we use --fuzz=0 to check patches apply cleanly for safety, due to the patch(1) issue detailed in commit v8.21-117-g46f7e05 * gl/lib/regcomp.c.diff: Rediffed. * gl/lib/regex_internal.c.diff: Likewise. * gl/lib/regex_internal.h.diff: Likewise. * gl/lib/regexec.c.diff: Likewise. * gl/lib/tempname.h.diff: Likewise. 2015-02-03 Bernhard Voelker tests: avoid a FP on aarch64 when comparing env/printenv output The following test fails on aarch64 on openSUSE's OpenBuildService due to glibc's execvp reversing the pointers of 'environ', i.e., the output of "env|tac" equals "env env" on that platform. * tests/misc/printenv.sh: Use 'env env' to work around the behavior on that platform. While at it, fix the grep pattern which suppressed all environment variables starting with an underscore "_" instead of "$_" (and "$LD_PRELOAD") only. 2015-02-02 Pádraig Brady dircolors: sync with Xiph file extensions * src/dircolors.hin: Remove deprecated anx, axa, axv. Add opus. Suggested by Scott Teal. 2015-01-31 Bernhard Voelker doc,maint: fix use of "i.e." in documentation and comments To align with all other places (and correct grammar), change all upper-case "I.E." to "I.e.". Furthermore, ensure that "i.e." is followed by a comma. Finally, ensure to use a double-space before "I.e.," at the beginning of a sentence. The following was used to change all offending uses (apart from old ChangeLog files): $ git grep -liF 'i.e.' \ | xargs sed -i \ -e 's/I\.E\./I.e./g' \ -e 's/\. \(I\.e\.\)/. \1/g' \ -e 's/\([Ii]\.e\.\)\( \)/\1,\2/g' \ -e 's/\([Ii]\.e\.\)$/\1,/g' * cfg.mk (sc_prohibit_uppercase_id_est): Add new rule. (sc_ensure_double_space_after_dot_before_id_est): Likewise. (sc_ensure_comma_after_id_est): Likewise. (old_NEWS_hash): Refresh hash via "make update-NEWS-hash". * NEWS: Change use of "id est" abbreviation via the above command. * README: Likewise. * README-prereq: Likewise. * doc/coreutils.texi: Likewise. * gl/lib/rand-isaac.c: Likewise. * gl/lib/tempname.c.diff: Likewise. * man/stdbuf.x: Likewise. * src/cat.c: Likewise. * src/copy.c: Likewise. * src/copy.h: Likewise. * src/cp.c: Likewise. * src/cut.c: Likewise. * src/dd.c: Likewise. * src/df.c: Likewise. * src/fiemap.h: Likewise. * src/longlong.h: Likewise. * src/ls.c: Likewise. * src/numfmt.c: Likewise. * src/pr.c: Likewise. * src/shred.c: Likewise. * src/shuf.c: Likewise. * src/split.c: Likewise. * tests/Coreutils.pm: Likewise. * tests/df/df-symlink.sh: Likewise. * tests/df/skip-rootfs.sh: Likewise. * tests/init.sh: Likewise. * tests/ls/color-norm.sh: Likewise. * tests/misc/basename.pl: Likewise. * tests/misc/ls-misc.pl: Likewise. * tests/misc/md5sum-bsd.sh: Likewise. * tests/misc/shred-exact.sh: Likewise. * tests/misc/sort.pl: Likewise. * tests/misc/stdbuf.sh: Likewise. * tests/misc/tac-continue.sh: Likewise. * tests/rm/r-root.sh: Likewise. * tests/tail-2/symlink.sh: Likewise. 2015-01-29 Pádraig Brady doc: clarify the output format for the *sum utilities * src/md5sum.c (usage): Detail the reasons for the default double space between checksum and file name. * doc/coreutils.texi (md5sum invocation): Likewise. Explicitly mention the 3 formats that --check supports. Fixes http://bugs.gnu.org/19725 2015-01-28 Giuseppe Scrivano sync: support syncing specified arguments * m4/jm-macros.m4 (coreutils_MACROS): Check for syncfs(). * man/sync.x: Add references to syncfs, fsync and fdatasync. * doc/coreutils.texi (sync invocation): Document the new feature. * src/sync.c: Include "quote.h". (AUTHORS): Include myself. (MODE_FILE, MODE_DATA, MODE_FILE_SYSTEM, MODE_SYNC): New enum values. (long_options): Define. (sync_arg): New function. (usage): Describe that arguments are now accepted. (main): Add arguments parsing and add support for fsync(2), fdatasync(2) and syncfs(2). * tests/misc/sync.sh: New (and only) test for sync. * tests/local.mk: Reference the new test. * AUTHORS: Add myself to sync's authors. * NEWS: Mention the new feature. 2015-01-25 Jim Meyering scripts: autotools-install: update * scripts/autotools-install: Increase automake's version number to 1.15 and add Stefano Lattarini's new GPG key ID. Increase gettext's version to 0.19.4 and add Daiki Ueno's GPG key ID. Also move VERSION definition "up" so that it is once again automatically updated via the emacs hook snippet at the end of the file. 2015-01-24 Pádraig Brady stty: only print supported options for combined options * src/stty.c (usage): Don't reference unsupported options, in the combined options descriptions. * doc/coreutils.texi (stty invocation): Adjust for the new order of the 'sane' and 'raw' combined options. Also add -iutf8 to the 'sane' list. stty: document the 'status' character where supported * src/stty.c (usage): On systems that support this setting (BSD), display 'status' in the list of adjustable special characters. * doc/coreutils.texi (stty invocation): Mention the option, and that it's not currently supported on Linux. 2015-01-24 Pádraig Brady stty: document the 'discard' character setting The equivalent of this is 'flush', but that was never documented as an option (though was output with stty -a). Therefore use the more descriptive name, also generally used on BSD systems. Note even though this setting seems ineffective on Linux, supporting the setting is useful to allow terminal programs to receive the default ^O character code. * doc/coreutils.texi (stty invocation): Document the 'discard' option. * src/stty.c (struct control_info): Add 'discard'; same as 'flush'. (display_all): Show 'discard' rather than 'flush' char. (display_changed): Likewise. (usage): Document the 'discard' option. 2015-01-24 Pádraig Brady stty: add support for extproc/LINEMODE Add support for the "extproc" option which is well described at: http://lists.gnu.org/archive/html/bug-readline/2011-01/msg00004.html * src/stty.c (usage): Describe the extproc option if either the Linux EXTPROC local option is defined, or the equivalent BSD TIOCEXT ioctl is defined. (main): Make the separate ioctl call for extproc on BSD. * doc/coreutils.texi (stty invocation): Describe the option, and reference the related RFC 1116. * NEWS: Mention the new feature. 2015-01-20 Bernhard Voelker tests: use returns_ function for new split test * tests/split/record-sep.sh: Use the recently added returns_ function to simplify the shell syntax in this test. Also remove the redirection of stdout/stderr to /dev/null as this eases analyzing errors. 2015-01-19 Assaf Gordon split: new -t option to select record separator * src/split.c (eolchar): A new variable to hold the separator character (unibyte for now). This is reference throughout rather than hardcoding '\n'. (usage): Describe the new --separator option, and mention records along with lines so there is no ambiguity that all options treat lines and records equivalently. (main): Have -t update eolchar, or default to '\n'. * tests/split/record-sep.sh: New test case. * tests/local.mk: Reference the new test. * doc/coreutils.texi (split invocation): Document the new option. Adjust --lines, --line-bytes, --number=[lr]/... to mention they pertain to records if --separator is specified. * NEWS: Mention the new feature. 2015-01-19 Pádraig Brady maint: fix typo in THANKS.in in previous commit * THANKS.in: s/Stehpen/Stephen/ 2015-01-19 Pádraig Brady doc: clarify that du operands are interdependent Following on from http://bugs.gnu.org/17546 make it more obvious that du may elide specified operands to avoid double counting in the set. * src/du.c (usage): Specify that du operates on the set of operands, rather than each independently. * doc/coreutils.texi (du invocation): Likewise. Also state that the number of entries printed may change due to the order specified. Currently, deeper items specified earlier will result in them being displayed, but don't mention that implementation detail in the documentation. * THANKS.in: Add reporter. Reported by Stephen Shirley 2015-01-14 Bernhard Voelker tests: use compare-vs-/dev/null instead of 'test -s' When some program produces unexpected output, that use of compare-vs-/dev/null will ensure that the surprising output is printed in the test's output. With "test -s err" only, one would have to instrument and rerun in order to see the offending output. * cfg.mk (sc_prohibit_and_fail_1): Exempt 'compare' from this check. * tests/dd/misc.sh: Change "tests -s ... || fail=1" to "compare /dev/null ... && fail=1". * tests/misc/nice.sh: Likewise. * tests/rm/read-only.sh: Likewise. * tests/tail-2/inotify-race.sh: Likewise. * tests/touch/no-dereference.sh: Likewise. Suggested by Jim Meyering in http://lists.gnu.org/archive/html/coreutils/2015-01/msg00042.html 2015-01-14 Pádraig Brady tests: add extra protection against unexpected exits Many tests use `program ... && fail=1` to ensure expected error situations are indicated. However that would mask an unexpected exit (like a crash). Therefore explicitly check the expected exit code. Note where error messages are also verified, the extra protection is not added. * tests/init.sh (returns_): A new helper function to check the return code of a command, and used throughout the tests. * cfg.mk (sc_prohibit_and_fail_1): Add a syntax check to avoid new instances of this issue. 2015-01-12 Pádraig Brady tests: avoid skipping some df tests with libmount * tests/df/no-mtab-status.sh: Provide libmount placeholders, to avoid skipping the test when libmount is in use. * tests/df/skip-duplicates.sh: Likewise. * tests/df/skip-rootfs.sh: Comment that the test is moot when libmount (/proc/self/mountinfo) is being used. 2015-01-10 Pádraig Brady maint: clean up some test issues identified with shellcheck * tests/cp/cp-a-selinux.sh: Comment why unused variables are assigned. Fix misspellings noticed while adjusting. * tests/cp/fiemap-perf.sh: Fix quoting. * tests/misc/shuf.sh: Avoid useless use of cat. * tests/misc/printf-surprise.sh: Likewise. 2015-01-08 Daiki Ueno maint: adjustments related to previous shuf crash fix * tests/misc/shuf.sh: Improve the test so it detects crashes in more cases. * NEWS: Mention the previous fix. 2015-01-08 Assaf Gordon maint: skip long-lines syntax-check if unsupported * cfg.mk(sc_long_lines): Skip if required wc and sed options are not supported, which is currently the case on OS X. 2015-01-06 Paul Eggert shuf: do not mishandle 'shuf -i0-0 1' Problem reported by Daiki Ueno in: http://bugs.gnu.org/19520 * src/shuf.c (main): Avoid core dump if !input_range. * tests/misc/shuf.sh: Test for this bug. 2015-01-01 Bernhard Voelker maint: update further copyright year number ranges * bootstrap: Update copyright year manually (missing in previous gnulib update). * tests/init.sh: Likewise. The entries in the exemption list are processed by "grep -vEf ./.x-update-copyright", and therefore evaluated as an extended regular expression (ERE). Thus, the "bootstrap" entry also matched for bootstrap.conf which we want to be updated. * .x-update-copyright: Change all entries to EREs, i.e. including the caret ^ and dollar sign $ meta-characters matching the beginning and the end of a line. * bootstrap.conf: Update copyright year by "make update-copyright". Finally, the only one showing up with the following command should be the COPYING file: $ git grep 'Copyright .* Free Software' | grep -v '2015 Free Software' 2015-01-01 Pádraig Brady doc: fix man page formatting for split CHUNKS options * src/split.c (usage): Indent the info on CHUNKS so that help2man can match it and align appropriately in its own section. Fixes http://bugs.gnu.org/19228 build: update to latest gnulib Pick up an errno adjustment in xstrtol() that fixes a spurious test failure on Darwin 14.0.0. Also update copyright year to 2015 avoiding a syntax-check failure. 2015-01-01 Pádraig Brady maint: update all copyright year number ranges Run "make update-copyright" and then... * tests/sample-test: Adjust to use the single most recent year. * tests/du/bind-mount-dir-cycle-v2.sh: Fix case in copyright message, so that year is updated automatically in future. 2014-12-30 Pádraig Brady tests: fix possible 8 minute running time of inotify-rotate.sh Commit v8.23-63-g111a2b9 removed the expensive tag on this test, as it runs quickly on systems with inotify. However without that it would take about 8 minutes for the test to complete all iterations. * tests/tail-2/inotify-rotate.sh: Tag as expensive without inotify. Also adjust the polling parameters used on systems without inotify so that the test completes within about 15 seconds. 2014-12-30 Pádraig Brady build: update to latest gnulib Included in this are gnulib changes 3ea43e02 2768ceb7 which make the device IDs from /proc/self/mountinfo available to df. This can be leveraged by a subsequent change to df to present a more accurate list of file systems. * bootstrap: Merge from gnulib. * src/ls.c (dev_ino_pop): s/obstack_blank/obstack_blank_fast/ as this API/ABI has changed, giving memory exhausted errors if negative (large positive) numbers are passed to obstack_blank(). * tests/df/skip-duplicates.sh: Adjust as the new gnulib code requires a non NULL mnt_opts even when mnt_type is not "none". 2014-12-29 Pádraig Brady maint: remove duplicate names from THANKS * .mailmap: Adjust so that there is only a single entry per name in the generated THANKS. 2014-12-26 Pádraig Brady doc: update the URL referencing SI prefixes Identified at https://www.gnu.org/software/gnun/linc/linc.html * doc/coreutils.texi (Block size): Fix the stale link. 2014-12-25 Pádraig Brady tests: fix error message check on some systems http://hydra.nixos.org/build/18129583 identified (on OS X) an incorrect test assumption in the previous commit. * gl/lib/xdectoint.c (__xnumtoint): Suppress the EINVAL error message as it's redundant in this context. * tests/misc/tail.pl: Suppress _optionally_ appended strerror messages. * tests/fmt/base.pl: Likewise. * tests/pr/pr-tests.pl: Likewise. * tests/split/l-chunk.sh: Likewise. 2014-12-19 Pádraig Brady diagnose too-large numbers better Following on from commit v8.23-82-gaddae94, consistently diagnose numbers that are too large, so as to distinguish from other errors, and make the limits obvious. * gl/modules/xdectoint: A new module implementing xdecto[iu]max(), which handles the common case of parsing a bounded integer and exiting with a diagnostic on error. * gl/lib/xdectoimax.c: The signed variant. * gl/lib/xdectoint.c: The parameterized implementation. * gl/lib/xdectoint.h: The interface. * gl/lib/xdectoumax.c: The unsigned variant. * bootstrap.conf: Reference the new module. * cfg.mk (exclude_file_name_regexp--sc_require_config_h_first): Exclude the parameterized templates. * src/csplit.c: Output EOVERFLOW or ERANGE errors if appropriate. * src/fmt.c: Likewise. * src/fold.c: Likewise. * src/head.c: Likewise. * src/ls.c: Likewise. * src/nl.c: Likewise. * src/nproc.c: Likewise. * src/shred.c: Likewise. * src/shuf.c: Likewise. * src/stdbuf.c: Likewise. * src/stty.c: Likewise. * src/tail.c: Likewise. * src/truncate.c: Likewise. * src/split.c: Likewise. * src/pr.c: Likewise. * tests/pr/pr-tests.pl: Adjust to avoid matching errno diagnostic. * tests/fmt/base.pl: Likewise. * tests/split/l-chunk.sh: Likewise. * tests/misc/shred-negative.sh: Likewise. * tests/misc/tail.pl: Likewise. Also remove the redundant existing ERR_SUBST from test err-6. * tests/ls/hex-option.sh: Check HEX/OCT options. * tests/misc/shred-size.sh: Likewise. * tests/misc/stty-row-col.sh: Likewise. 2014-12-19 KO Myung-Hun build: add $(EXEEXT) suffix to man page make targets * man/local.mk: Add $(EXEEXT) suffix to the executables, which is significant on OS/2 for example. build: don't call OS/2 routines on all systems * src/system.h: Add a missing __OS2__ ifdef guard. Also adjust spacing around () to avoid a syntax-check failure. 2014-12-18 KO Myung-Hun build: expand a response file and a wildcard on OS/2 OS/2 traditional shells(cmd) do not expand a response file(@file) or a wildcard. Expand them in each utility itself. * src/system.h (initialize_main): Define on OS/2. Expand a response file and a wildcard. 2014-12-16 Bernhard Voelker maint: fix how to request changes for THANKS.in * THANKS.in: Change the comment at the top to send change requests regarding this file to the main mailing list rather than referring to cp's --help output for the mailing list's address - which does not include that information anymore. 2014-12-16 Paul Eggert dd: fix typo in previous change Reported by Bernhard Voelker in: http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00191.html * src/dd.c (scanargs): s/IN/OUT/. dd: diagnose too-large numbers better Reported by Isabella Parakiss in: http://lists.gnu.org/archive/html/bug-gnulib/2014-12/msg00184.html * src/dd.c (parse_integer): Return strtol_error code, not bool. All callers changed. (scanargs): Improve quality of diagnostic when a number is too large. 2014-12-10 Pádraig Brady build: use the system printf routines in most cases glibc <= 2.5 would crash when passed invalid long double values, therefore internal gnulib routines were used, essentially only by od, to output such invalid values. Later glibc versions don't crash, as per https://sourceware.org/bugzilla/show_bug.cgi?id=4586 and subsequently od was adjusted to use the system printf routines through the use of the ftoastr module with commit v8.7-22-ga71c22f. Consequently our testing of this feature was moot, and use of the gnulib printf replacement for printf(1), od(1) and error(3) etc. was redundant. * configure.ac (gl_printf_safe): Unset so that we don't check that "nan" is output for these long double values. * tests/misc/od-float.sh: Adjust all existing checks to fail if od exits with failure status (like crashing for example). Add a new case for one of the problematic invalid long double values for x86_64. We only check that od exits successfully at present, which may change if https://sourceware.org/bugzilla/show_bug.cgi?id=17661 is resolved. 2014-12-03 Pádraig Brady maint: avoid -fsanitize=undefined warning in rand-isaac * gl/lib/rand-isaac.c (isaac_refill): readisaac() purposefully passes unaligned pointers to avoid memory copies. This is only done on platforms where this is defined, so avoid the associated runtime warning generated with -fsanitize=undefined, which is: lib/rand-isaac.c:125:182: runtime error: store to misaligned address 0x63100003d7fd for type 'isaac_word', which requires 8 byte alignment 0x63100003d7fd: note: pointer points here 47 ce ed a4 be be be 00 00 00 00 00 00 00 00 ... ^ 2014-12-02 Pádraig Brady maint: avoid signed overflow warning with -O3 Prompted by the implicit -O3 added by american-fuzzy-lop, seen with GCC 4.9.2 on x86_64. src/pr.c: In function 'print_files.part.5': src/pr.c:1781:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow] if (cols_ready_to_print () == 0) This happens because cols_ready_to_print() is inlined thus reducing the comparison to the N variable in print_page(). Now this can't overflow due to the protection when parsing the specified column, but use an unsigned type to avoid the apparent signed overflow. * src/pr.c (cols_ready_to_print): Increment an unsigned type to avoid the subsequent signed overflow warning. 2014-12-02 Boris Ranto du: handle sub-bind-mount cycles gracefully This patch fixes the handling of sub-bind-mount cycles which are incorrectly detected as the file system errors. If you bind mount the directory 'a' to its subdirectory 'a/b/c' and then run 'du a/b' you will get the circular dependency warning even though nothing is wrong with the file system. This happens because the first directory that is traversed twice in this case is not a bind mount but a child of bind mount. The solution is to traverse all the directories in the cycle that fts detected and check whether they are not a (bind) mount. * src/du.c (mount_point_in_fts_cycle): New function that checks whether any of the directories in the cycle that fts detected is a mount point. * src/du.c (process_file): Update the function to use the new function that looks up all the directories in the fts cycle instead of only the last one. * tests/du/bind-mount-dir-cycle-v2.sh: New test case that exhibits the described behavior. * tests/local.mk: Reference the new root test. * NEWS: Mention the bug fix. 2014-11-29 Pádraig Brady tests: avoid hardlink to symlink tests where not supported These checks weren't correctly avoided in commit v8.23-66-g222d7ac * tests/cp/same-file.sh: Avoid all hardlink to symlink tests on platforms where that's not supported. Identified by http://hydra.nixos.org/build/17636446 2014-11-28 Pádraig Brady build: fix missing casts from recent change * src/dd.c (alloc_[io]buf): I committed a stale patch that omitted the casts needed on 32 bit. Identified by http://hydra.nixos.org/build/17610188 2014-11-27 Pádraig Brady rm: fix prompted number of arguments to remove on some platforms "zu" was output on solaris 8 for example rather than the number, since coreutils-8.22. * cfg.mk: Disallow %z, since we don't currently use the gnulib fprintf module, so any usage with it is non portable. Also our usage with error() currently works only through an ancillary dependency on the vfprintf gnulib module. * src/rm.c (main): Use %PRIuMAX rather than %zu for portability. * src/dd.c (alloc_[io]buf): Likewise for consistency. * src/od.c (main): Likewise. * src/split.c (set_suffix_length): Likewise. * NEWS: Mention the rm bug fix. Reported in http://bugs.gnu.org/19184 2014-11-26 Pádraig Brady tests: fix portability issue in dd/ascii test Solaris 8 was seen to issue this error: "printf: `&': illegal format character" * test/dd/ascii.sh: Use the coreutils printf in this test rather than the system one, to avoid portability issues. 2014-11-26 Paul Eggert build: port new rule for coreutils.h to old Bash Reported by Ted Carr in: http://bugs.gnu.org/19184 * src/local.mk (src/coreutils.h): Don't assume single_binary_progs is nonempty. 2014-11-24 Tobias Stoeckmann paste: fix possible truncated output with large files If '\n' was present at the size_t boundary of a file, then that and subsequent data would be discarded. * src/paste.c (paste_parallel): Avoid the overflow issue by changing the flag to a boolean rather than a count. * NEWS: Mention the bug fix. 2014-11-24 Pádraig Brady df: only suppress remote mounts of separate exports with --total * src/df.c (filter_mount_list): Separate remote locations are generally explicitly mounted, so list each even if they share the same remote device and thus storage. However with --total keep the suppression to give a more accurate value for the total storage available. (usage): Expand on the new implications of --total and move it in the options list according to alphabetic order. doc/coreutils.texi (df invocation): Mention that --total impacts on deduplication of remote file systems and also move location according to alphabetic order. * tests/df/skip-duplicates.sh: Add remote test cases. * NEWS: Mention the change in behavior. Reported in http://bugs.debian.org/737399 Reported in http://bugzilla.redhat.com/920806 Reported in http://bugzilla.opensuse.org/866010 Reported in http://bugzilla.opensuse.org/901905 2014-11-23 Pádraig Brady df: ensure -a shows all remote file system entries commit v8.22-125-g9d736f8 printed placeholder "-" values for device names that didn't match the preferred device name for a particular mount point. However that was seen to erroneously suppress values for aliased host names or exports, common with remote file systems. * src/df.c (me_for_dev): Rename from devname_for_dev() so that we can determine the remoteness as well as the name for the preferred mount entry. (get_dev): Don't output place holder values when both current and preferred mount entries are remote. Reported in http://bugs.debian.org/737399 2014-11-22 Pádraig Brady tests: add a case verifying mv on case insensitive file systems * NEWS: Update the recent entry to also mention the avoidance of incorrectly unlinking a multi-hardlinked "source" file when presented with source and dest that only differ in case. * src/copy.c (same_file_ok): Mention the case issue with same_name(). * tests/mv/hardlink-case.sh: Test the issue on HFS+. * tests/local.mk: Reference the new test case. * tests/mv/vfat: Remove an old related but unused test case. 2014-11-21 Pádraig Brady doc: mention how to avoid newlines impacting ls -1 * src/ls.c (usage): Mention the -b and -q options in the -1 description. * doc/coreutils.texi (ls invocation): Likewise. 2014-11-21 Pádraig Brady tests: chcon: avoid false failure with newer selinux file_t is now mapped to unlabeled_t as per: http://danwalsh.livejournal.com/68189.html Therefore use the latter to ensure we match correctly. This is needed on >= Fedora 21 for example, while it also works on earlier releases. 2014-11-21 Boris Ranto mv: fail when moving a file to a hardlink We may run into a race condition if we treat hard links to the same file as distinct files. If we do 'mv a b' and 'mv b a' in parallel, both a and b can disappear from the file system. The reason is that in this case the unlink on src is called and the system calls can end up being run in the order where unlink(a) and unlink(b) are the last two system calls. Therefore exit with an error code so that we avoid the potential data loss. * src/copy.c (same_file_ok): Don't set unlink_src that was used by mv, and return false for two hardlinks to a file in move_mode. *src/copy.c (copy_internal): No longer honor the unlink_src option, used only by mv. NEWS: Mention the change in behavior. * tests/cp/same-file.sh: Augment to cover the `cp -a hlsl1 sl1` case. * tests/mv/hard-verbose.sh: Remove no longer needed test. * tests/local.mk: Remove the reference to hard-verbose.sh. * tests/mv/hard-4.sh: Adjust so we fail in this case. * tests/mv/i-4.sh: Likewise. * tests/mv/symlink-onto-hardlink-to-self.sh: Likewise. 2014-11-18 Pádraig Brady maint: run strftime syntax check on newer systems * cfg.mk (sc_strftime_check): Adjust regex to handle newer glibc info formatting with different indentation and quoting. 2014-10-30 Pádraig Brady maint: tests: fix comments about retry_delay_ * tests/tail-2/F-vs-missing.sh: Comment with the correct total delay. * tests/tail-2/F-vs-rename.shi: Likewise. * tests/tail-2/flush-initial.sh: Likewise. * tests/tail-2/inotify-hash-abuse.sh: Likewise. * tests/tail-2/pipe-f2.sh: Likewise. * tests/misc/chroot-fail.sh: Initialize can_chroot_root in all cases. tests: make inotify-rotate more robust and efficient * tests/tail-2/inotify-rotate.sh: Use retry_delay_ to employ an exponential backoff with a total delay of up to 25.5s. The 15s delay was seen to trigger a false failure in http://hydra.nixos.org/build/16546517 Also remove the .1s sleep in each of the 50 iterations to reduce the running time of the test and thus the expensive_ tag on this test was removed. Also ensure that we use the standard exit procedure upon failure to avoid any erroneous diagnostics due to persistent files on NFS. 2014-10-29 Bernhard Voelker tests: make du/move-dir-while-traversing more robust * tests/du/move-dir-while-traversing.sh: Catch failure of retry_delay_ when waiting for the watcher to get ready. 2014-10-27 Pádraig Brady doc: mention that df -a includes duplicate file systems * src/df.c (usage): Mention that duplicate file systems are shown with this option, not just dummy file systems. * doc/coreutils.texi (df invocation): For the --all option, expand on the class of normally suppressed mount entries that it includes. Reported in http://bugs.debian.org/737399 2014-10-23 Pádraig Brady maint: improve sc_long_lines syntax-check speed sc_long_lines was the slowest syntax check before$ time make sc_long_lines long_lines real 0m2.740s after $ time make sc_long_lines long_lines real 0m0.677s * src/cfg.mk (sc_dd_max_sym_length): s/--max-line-length/-L/ for compat with BSDs. (sc_long_lines): Prefilter with wc -L to only identify lines in files that have lines longer than 80 characters. 2014-10-23 Mike Frysinger tests: d_type-check: don't hardcode the C library name * tests/d_type-check: The hardcoded name doesn't hold true for all Linux/glibc platforms, let alone Linux/non-glibc. Use ctypes.util.find_library() instead to search for the library. 2014-10-17 Bernhard Voelker tests: avoid false failure when comparing /proc files At least the MHz number in /proc/cpuinfo may change, thus leading to a false positive failure when comparing the expected against the actual output file. Use an invariant file instead: /proc/version. * tests/misc/head-c.sh: s/cpuinfo/version/ 2014-10-16 Pádraig Brady tests: fix test hang with unstable inodes in /proc * cp/proc-zero-len.sh: Search the 'err' file for the error to ignore, not stdin. chroot: call chroot() unconditionally to handle bind mounted "/" * src/chroot.c (is_root): Adjust to compare canonicalized paths rather than inodes, to handle (return false in) the case where we have a tree that is constructed by first bind mounting "/" (thus having the same inode). (main): Unconditionally call chroot() because it's safer and of minimal performance benefit to avoid in this case. This will cause inconsistency with some platforms not allowing `chroot / true` for non root users. * tests/misc/chroot-fail.sh: Adjust appropriately. * NEWS: Mention the bug fixes. Fixes http://bugs.gnu.org/18736 2014-10-15 Pádraig Brady copy: avoid an extraneous error when reporting errors * src/copy.c (copy_reg): If sparse_copy() failed, then an erroneous error about failing to extend the file would be reported. cp: read sparse files more efficiently with non regular destination * src.copy.c (copy_reg): Use fiemap to read sparse files, even if the output is not to a regular file. * NEWS: Mention the improvement. 2014-10-15 Pádraig Brady cp: avoid speculative preallocation with --sparse=always With --sparse=always use fallocate(...PUNCH_HOLE...) to avoid any permanent allocation due to speculative preallocation employed by file systems such as XFS. * m4/jm-macros.m4: Check for and fallocate(). * src/copy.c (punch_hole): A new function to try and punch a hole at the specified offset if supported. (create_hole): Call punch_hole() after requesting a hole. (extent_copy): Likewise. * NEWS: Mention the improvement. 2014-10-15 Pádraig Brady copy: detect smaller holes than the copy buffer size Previously cp would not detect runs of NULs that were smaller than the buffer size used for I/O (currently 128KiB). * src/copy.c (copy_reg): Use an independent hole_size, set to st_blksize, to increase the chances of detecting a representable hole, in a run of NULs read from the input. (create_hole): A new function refactored from sparse_copy() and extent_copy() so we have a single place to handle holes. (sparse_copy): Adjust to loop over the larger input buffer in chunks of the passed hole size. Also adjust to only call lseek once per hole, rather than at least once per input buffer. * tests/cp/sparse.sh: Add test cases for various sparse chunk sizes. * NEWS: Mention the improvement. 2014-10-09 Wieland Hoffmann doc: clarify that timeout limits are not 2038 seconds * man/timeout.x: The 2038 that the sentence is referring to is the year 2038, not 2038 seconds (the default unit used for timeouts). 2014-10-08 Pádraig Brady maint: avoid new signed overflow warning on 32 bit Prompted by http://hydra.nixos.org/build/15682577 with GCC 4.8.3 on i686 src/tac.c:557:6: error: assuming signed overflow does not occur when simplifying conditional to constant [-Werror=strict-overflow] if (bytes_copied < 0) This happens because copy_to_temp() is inlined in tac_nonseekable(), thus reducing the comparison to the bytes_copied variable in copy_to_temp. Now this can't overflow on either 32 or 64 bit due to the protection of the preceding fwrite(). We could use a guard like "if (bytes_copied <= OFF_T_MAX - bytes_read)" to avoid the warning, but rather than a runtime branch, just use an unsigned type to avoid apparent signed overflow on systems where the accumulation is not promoted to unsigned (32 bit size_t, 64 bit off_t). * src/tac.c (copy_to_temp): Increment an unsigned type to avoid the subsequent signed overflow warning. 2014-10-08 Jim Meyering tests: split/b-chunk.sh: avoid spurious fail on non-Linux * tests/split/b-chunk.sh: Skip each file that does not exist. Some systems lack /proc/version or /sys/kernel/profiling 2014-10-08 Pádraig Brady maint: avoid syntax-check failures in previous patch * tests/misc/od-j.sh: Non standard comparison order. * tests/split/b-chunk.sh: Confusing input file name. * tests/tail-2/tail-c.sh: Redundant require ulimit. 2014-10-08 Paul Eggert wc: don't miscount /sys and similar file systems Fix similar problems in head, od, split, tac, and tail. Reported by George Shuklin in: http://bugs.gnu.org/18621 * NEWS: Document this. * src/head.c (elseek): Move up. (elide_tail_bytes_pipe, elide_tail_lines_pipe): New arg CURRENT_POS. All uses changed. (elide_tail_bytes_file, elide_tail_lines_file): New arg ST and remove arg SIZE. All uses changed. * src/head.c (elide_tail_bytes_file): * src/od.c (skip): Avoid optimization for /sys files, where st_size is bogus and st_size == st_blksize. Don't report error at EOF when not optimizing. * src/head.c, src/od.c, src/tail.c: Include "stat-size.h". * src/split.c (input_file_size): New function. (bytes_split, lines_chunk_split, bytes_chunk_extract): New arg INITIAL_READ. All uses changed. Use it to double-check st_size. * src/tac.c (tac_seekable): New arg FILE_POS. All uses changed. (copy_to_temp): Return size of temp file. All uses changed. * src/tac.c (tac_seekable): * src/tail.c (tail_bytes): * src/wc.c (wc): Don't trust st_size; double-check by reading. * src/wc.c (wc): New arg CURRENT_POS. All uses changed. * tests/local.mk (all_tests): Add tests/misc/wc-proc.sh, tests/misc/od-j.sh, tests/tail-2/tail-c.sh. * tests/misc/head-c.sh: * tests/misc/tac-2-nonseekable.sh: * tests/split/b-chunk.sh: Add tests for problems with /proc and /sys files. * tests/misc/od-j.sh, tests/misc/wc-proc.sh, tests/tail-2/tail-c.sh: New files. 2014-10-04 Bernhard Voelker doc: document stat's output with the --terse option * doc/coreutils.texi (stat invocation): Add a paragraph documenting stat's output format when the --terse option is specified, both in normal and in --file-system mode. Reported by Dan Jacobson in http://bugs.gnu.org/18624 2014-10-02 Nick Alcock tests: fix false failure for test referencing libdl * init.cfg (gcc_shared_): -ldl has to be positioned after the object files that may rely upon it. This fixes tests/cp/nfs-removal-race.sh which references dlsym() from libdl. 2014-10-02 Bernhard Voelker maint: avoid double semicolon syntax check failure A syntax-check recently added to gnulib would trigger a failure (once gnulib gets updated here) for a statement introduced with commit v8.23-43-gaf2a4ed: src/dd.c:806: char const *time_fmt = _(", %g s, %s/s\n");; maint.mk: Double semicolon detected make: *** [sc_prohibit_double_semicolon] Error 1 * src/dd.c (print_xfer_stats): s/;;/;/ 2014-09-30 Federico Simoncelli dd: new status=progress level to print stats periodically * src/dd.c: Report the transfer progress every second when the new status=progress level is used. Adjust the handling and description of the status= option so that they're treated as mutually exclusive levels, rather than flags with implicit precedence. * doc/coreutils.texi (dd invocation): Document the new progress status level. Reference the new level in the description of SIGUSR1. * tests/dd/stats.sh: Add new test for status=progress. * tests/dd/misc.sh: Change so status=none only takes precedence if it's the last level specified. * NEWS: Mention the feature. 2014-09-30 Pádraig Brady dd: use more robust SIGUSR1 handling * src/dd.c (ifd_reopen): A new wrapper to ensure we don't exit upon receiving a SIGUSR1 in a blocking open() on a fifo for example. (iftruncate): Likewise for ftruncate(). (iread): Process signals also after a short read. (install_signal_handlers): Install SIGINFO/SIGUSR1 handler even if set to SIG_IGN, as this is what the parent can easily set from a shell script that can send SIGUSR1 without the possiblity of inadvertently killing the dd process. * doc/coreutils.texi (dd invocation): Improve the example to show robust usage wrt signal races and short reads. * tests/dd/stats.sh: A new test for various signal races. * tests/local.mk: Reference the new test. * NEWS: Mention the fix. 2014-09-24 Paul Eggert test: check for Fedora 20 sort key bug Problem reported by Göran Uddeborg in: http://bugs.gnu.org/18540 * tests/misc/sort.pl: New test 23. 2014-09-24 David Sterba mv: use reflink=auto mode by default On some filesystems (BTRFS), moving a file within the filesystem may cross subvolume boundaries and we can use a lightweight reflink copy, similar to what cp(1) can do, which is faster than a full file copy. This is enabled by default because it's only an optimization for the fall back copy and does not break user expectations or usability. * src/mv.c (cp_option_init): Set the reflink mode to AUTO. * NEWS: Mention the improvement. 2014-09-23 Pádraig Brady stty: only list supported options in --help and man pages * src/stty.c (usage): Exclude unsupported options from --help, which for example impacts the "dsusp" and "cdtrdsr" options on Linux. Fixes http://bugs.gnu.org/18506 2014-09-23 Michal Nazarewicz doc: fix use of "e.g." in stdbuf help message "E.g." stands for latin "exempli gratia" which is typically read as "for example". "E.g." does not stand for the word "example". As such, "for e.g." might be read as "for for example". Fix this usage by simply replacing "e.g." with "example". 2014-09-23 Bernhard Voelker build: pass program name to help2man after other options * man/local.mk (.x.1): Move the program name argument down after the last option argument when calling $(run_help2man). While the other way would be accepted for the GNU help2man program, it is not for the 'dummy-man' script (called as a fallback on systems lacking perl). The wrong order was introduced in commit v8.21-119-gb3578fc while adding the --info-page option. build: fix argument count check in dummy-man again * man/dummy-man: Fix argument count check, now only permitting exactly 1 argument, the program name. Reported by Andreas Schwab 2014-09-22 Alban Bedel build: fix an inverted test breaking dummy-man * man/dummy-man: Fix argument count check, allowing dummy-man to run (on systems without perl). Fixes http://bugs.gnu.org/18531 2014-09-19 Pádraig Brady build: fix dependency issues with man page generation * .gitignore: Remove reference to no longer generated make file. * configure.ac: Don't bother generating placeholder make file. * man/local.mk: Hardcode the man page deps list for normal builds to be compatible with all make implementations and configure options. Note in SINGLE_BINARY mode, all man pages will be generated on any change to the coreutils binary, but development will generally not be done in this mode, so this shouldn't be an issue. Fixes http://bugs.gnu.org/18055 2014-09-19 Jim Meyering maint: don't trigger gcc-5's new -Wlogical-not-parentheses warning * src/dircolors.c (main): Parenthesize !VAR as LHS to "<", to avoid triggering gcc's new -Wlogical-not-parentheses warning. 2014-09-19 Pádraig Brady cp: fix handling of -H with multiply specified source dirs Following on from commit v5.92-729-g130dd06, also avoid the erroneous directory hardlink warning with -H. * src/copy.c (copy_internal): Also handle the -H case for command line arguments. * tests/cp/duplicate-sources.sh: Augment the test case. * NEWS: Augment the news entry. 2014-09-19 Pádraig Brady cp: issue correct warning and ignore duplicate source dirs * src/copy.c (copy_internal): Handle the case where we have the same destination directory as already encountered, which can only be due to the corresponding source directory being specified multiple times. * tests/cp/duplicate-sources.sh: Add a test for the new multiply specified directory case, and the existing multiply specified file case. * tests/local.mk: Reference the new test. * NEWS: Mention the bug fix. doc: output correct --help references with --program-prefix * src/system.h (emit_ancillary_info): Take the invariant PROGRAM_NAME as a parameter, so that consistent references are made to online docs and texinfo nodes, when a --program-prefix is in place. Note the man pages don't need this fix as they're generated before the program prefix is used. * NEWS: Mention the improvements in references to online documentation. doc: ensure the correct texinfo nodes are referenced in --help * src/system.h (emit_ancillary_info): For commands that don't have a 1:1 mapping with the texinfo node names, provide a mapping to the correct node. * doc/coreutils.texi: Add some extra cross references noticed while checking this. Fixes http://bugs.debian.org/762092 2014-09-16 Bernhard Voelker tests: fix false du failure on newer XFS On XFS, when creating the ~2G test file 'big' in a for-loop by appending 20M each time, the file ends up using ~4G - visible in 'st_blocks'. The unused space would be reclaimed later. This feature is called "speculative preallocation" which aims at avoiding fragmentation. According to the XFS FAQ [1], there are two particular aspects of XFS speculative preallocation that are triggering this: 1. "Applications that repeatedly trigger preallocation and reclaim cycles [after file close] can cause fragmentation. Therefore, this pattern is detected and causes the preallocation to persist beyond the lifecycle of the file descriptor." 2. "Preallocation sizes grow as files grow larger." [1] http://xfs.org/index.php/XFS_FAQ Avoid one of the above by only doing a single close (reclaim cycle). * tests/du/2g.sh: Similar to the fix for a dd test (see commit v8.22-65-g7c03fe2), avoid speculative preallocation by creating the 'big' file in one go instead of appending to it in the loop. Remove debugging statements as the output with 'set -x' is sufficient nowadays. 2014-09-11 Paul Eggert cat: allow copying empty files to themselves Problem reported by Vincent Lefevre in: http://bugs.gnu.org/18449 * src/cat.c (main): Allow copying an empty file to itself. * tests/misc/cat-self.sh: New test. * tests/local.mk (all_tests): Add it. 2014-09-11 Pádraig Brady doc: reference online info pages directly from man pages * src/system.h (emit_ancillary_info): Add a direct reference to the corresponding online info documentation. Corresponding redirects were put in place on www.gnu.org to allow for concise links. * help2man: Adjust to add the "online help" link (and subsequent translation bugs link) to a "REPORTING BUGS" section. Also add the concise links for further information in --help to the "SEE ALSO" section, and dispense with the more verbose default for that. 2014-09-10 Pádraig Brady doc: adjust reference to info nodes in man pages old form: coreutils '$cmd invocation' new form: '(coreutils) $cmd invocation' The old form erroneously referenced the node for the 'coreutils' multi-call program. Now that problematic node name was renamed in commit v8.23-18-g72e470b, but the newer less ambiguous form also has the advantage of working with the pinfo viewer for example. Full discussion at http://bugs.gnu.org/18428 * man/local.mk: Adjust man page references to texinfo nodes. * src/system.h: Adjust --help references to texinfo nodes. 2014-09-10 Pádraig Brady maint: include libstdbuf.c in extraneous headers check * cfg.mk (sc_system_h_headers): Don't exclude libstdbuf.c. * src/libstdbuf.c: Remove headers already included in system.h. build: adjust previous transformations on libstdbuf name * src/local.mk (transform): commit v8.23-22-g6f9b018 discarded all transformations on the libstdbuf.so name. Be more conservative and only exclude the $(program_transform_name) portion for libstdbuf. build: avoid name transformations on libstdbuf * src/local.mk (transform): Skip the transformation for libstdbuf since that should not be subject to name clashes, and we need to reference the name directly in LD_PRELOAD etc. * configure.ac: Add a comment on the coupling of pkglibexec_PROGRAMS to $(transform). Issue reported at https://trac.macports.org/ticket/44922 Improved by Nick Bowler 2014-09-09 Pádraig Brady maint: fix syntax-check issues in recent commit Avoid 2 new syntax-check failures introduced in commit v8.23-19-g8defcee * cfg.mk (sc_some_programs_must_avoid_exit_failure): s/exit/return/. * src/whoami.c (main): Reinstate translation marker for diagnostic. 2014-09-09 Paul Eggert maint: avoid file-scope names of the form _[a-z]* The C standard says this isn't portable, if you include standard include files. * build-aux/gen-single-binary.sh: * src/coreutils-arch.c (single_binary_main_arch) (single_binary_main_uname): * src/coreutils-dir.c (single_binary_main_ls) (_single_binary_main_dir): * src/coreutils-vdir.c (single_binary_main_ls) (_single_binary_main_vdir): * src/coreutils.c (SINGLE_BINARY_PROGRAM): Remove leading _ from single_binary prefix. * src/numfmt.c (round_style): Rename from _round. All uses changed. (inval_style): Rename from _invalid. All uses changed. maint: prefer 'return status;' to 'exit (status);' in 'main' * build-aux/gen-single-binary.sh: Don't use ATTRIBUTE_NORETURN for main functions. * src/base64.c, src/basename.c, src/cat.c, src/chcon.c, src/chgrp.c: * src/chmod.c, src/chown.c, src/chroot.c, src/cksum.c, src/comm.c: * src/cp.c, src/csplit.c, src/cut.c, src/date.c, src/dd.c, src/df.c: * src/dircolors.c, src/dirname.c, src/du.c, src/echo.c, src/env.c: * src/expand.c, src/expr.c, src/factor.c, src/fmt.c, src/fold.c: * src/getlimits.c, src/groups.c, src/head.c, src/hostid.c: * src/hostname.c, src/id.c, src/install.c, src/join.c, src/kill.c: * src/link.c, src/ln.c, src/logname.c, src/ls.c, src/make-prime-list.c: * src/md5sum.c, src/mkdir.c, src/mkfifo.c, src/mknod.c, src/mktemp.c: * src/mv.c, src/nice.c, src/nl.c, src/nohup.c, src/nproc.c: * src/numfmt.c, src/od.c, src/paste.c, src/pathchk.c, src/pinky.c: * src/pr.c, src/printenv.c, src/printf.c, src/ptx.c, src/pwd.c: * src/readlink.c, src/realpath.c, src/rm.c, src/rmdir.c, src/runcon.c: * src/seq.c, src/shred.c, src/shuf.c, src/sleep.c, src/sort.c: * src/split.c, src/stat.c, src/stdbuf.c, src/stty.c, src/sum.c: * src/sync.c, src/tac.c, src/tail.c, src/tee.c, src/timeout.c: * src/touch.c, src/tr.c, src/true.c, src/truncate.c, src/tsort.c: * src/tty.c, src/uname.c, src/unexpand.c, src/uniq.c, src/unlink.c: * src/uptime.c, src/users.c, src/wc.c, src/who.c, src/whoami.c: In 'main' functions, Prefer 'return status;' to 'exit (status);'. * src/coreutils-arch.c (_single_binary_main_uname) (_single_binary_main_arch): * src/coreutils-dir.c, src/coreutils-vdir.c (_single_binary_main_ls) (_single_binary_main_dir, _single_binary_main_vdir): Omit ATTRIBUTE_NORETURN. Return a value. * src/coreutils.c (SINGLE_BINARY_PROGRAM): Omit ATTRIBUTE_NORETURN. (launch_program): Now static. * src/dd.c (finish_up): New function. (quit, main): Use it. * src/getlimits.c (main): Return a proper exit status. * src/test.c (test_main_return): New macro. (main): Use it. * src/logname.c, src/nohup.c, src/whoami.c: Use 'error' to simplify exit status in 'main' function. * src/yes.c (main): Use 'return' rather than 'error' to exit, so that GCC doesn't suggest ATTRIBUTE_NORETURN. doc: rename "coreutils invocation" to "Multi-call invocation" This supports longstanding shell commands like 'info coreutils "touch invocation"'. Problem reported by Vincent Lefevre via Bob Proulx in: http://bugs.gnu.org/18428 * doc/coreutils.texi (Multi-call invocation): Rename from "coreutils invocation". doc: mention which commands are optional * doc/coreutils.texi (coreutils invocation, df invocation) (stty invocation, whoami invocation, nproc invocation) (arch invocation, hostname invocation, hostid invocation) (uptime invocation, chroot invocation, nice invocation) (stdbuf invocation): Document that the command is installed optionally. 2014-09-07 Bernhard Voelker doc: fix typos * doc/coreutils.texi: Fix normal typos: s/pseudorandom/pseudo-random/; s/behaviour/behavior/; s/linux-based/Linux-based/; s/nonnegative/non-negative/. Fix IEC's long name: s/Electronical/Electrotechnical/. Wrap 'getopt' into the @code{} macro. Fix a grammatical error (from myself): s/splitted/split/. 2014-09-03 Paul Eggert doc: spell "indeterminate" correctly * doc/coreutils.texi (timeout invocation): Fix misspelling. Reported by Yureruchihirosan via OKANO Takayoshi in: http://bugs.gnu.org/18394 2014-08-23 Pádraig Brady maint: ensure fiemap extents flags are compared correctly * src/extent-scan.c (extent_scan_read): Following on from the flags size adjustment in commit v8.23-13-g1505b37, verify that the internal representation of the flags is never truncated which could happen in the unlikely case on 32 bit if the kernel flags ever expanded to 64 bits which is theoretically possible given the reserved space. 2014-08-22 Paul Eggert maint: avoid int64_t and similar types unless they're needed C11 doesn't require them, even POSIX doesn't strictly require the 64-bit versions, and it makes the code a bit clearer if they're used only when needed. * src/copy.c (write_zeros, extent_copy): * src/extent-scan.h (struct extent_info.ext_length): Use off_t, not uint64_t, for a value derived from a file offset. * src/extent-scan.h (struct extent_info.ext_flags) Prefer plain unsigned int to uint32_t where either will do. (struct extent_scan.ei_count): Use size_t, not uint32_t, for a value bounded by SIZE_MAX. * src/factor.c (MAGIC64, MAGIC63, MAGIC65): Remove unnecessary casts to uint64_t. 2014-08-21 Yurij Goncharuk maint: refactor ls QUOTING_STYLE env var handling * src/ls.c (main): As per the FIXME comment, move the QUOTING_STYLE handling to a separate function. 2014-08-21 Fridolin Pokorny doc: clarify that duplicate NFS mounts are skipped by df * doc/coreutils.texi (df invocation): Add a sentence that eliding duplicate entries for the same file system is not limited to bind mounts, but also happens for remote file systems like NFS. 2014-08-19 Pádraig Brady df: improve mount point selection with inaccurate mount list v8.23 has a test failure on Fedora rawhide build servers in tests/df/skip-duplicate.sh. This was due to no '/' entry being output by df. That was due to an inaccurate /proc/mounts on the build environment as stat(/mnt/point) identified all these /proc/mounts entries as having the same device id: / rootfs / /dev/md1 /dev devtmpfs /run tmpfs /boot /dev/md0 /proc/filesystems /dev/md1 Since the device name on the right changes for a given id, that causes the entries to be continually replaced, thus resulting in no '/' entry. I'm guessing this is due to the mock environment bind mounting unneeded or sensitive items to a dummy file on the host / (/dev/md1) though have not looked into those details. So rather than relying on an accurate /proc/mounts, the attached patch takes a more conservative replacement approach and only swaps a new device entry when the mount point matches. That should handle all practical cases while also avoiding this situation. * src/df.c (filter_mount_list): Only replace entries with different device names when the mount point also matches. 2014-08-11 Rasmus Borup Hansen cp: remove redundant possibly expensive heap deallocation If the hash structures grow sufficiently large so that the system is actively swapping, then the deallocation can take a significant amount of time. Details at: http://lists.gnu.org/archive/html/coreutils/2014-08/msg00012.html * src/cp.c (main): Only call hash deallocation routines when in lint checking mode. * THANKS.in: Remove as now in the git author list. 2014-08-04 Reuben Thomas doc: indicate that FILE arguments are optional with rm -f * src/rm.c (usage): s/FILE/[FILE]/. Fixes http://bugs.gnu.org/18187 2014-08-03 Anders Jonsson numfmt: fix misspelling in --debug message * src/numfmt.c (parse_format_string): s/overridding/overriding/. Fixes http://bugs.gnu.org/18050 2014-08-01 Paul Eggert tests: fix typo in tail-2/inotify-race Reported by Andreas Schwab in: http://bugs.gnu.org/18057 * tests/tail-2/inotify-race.sh (break_src): Use abs_top_srcdir, not abs_top_builddir. 2014-08-01 Bernhard Voelker chroot: perform chdir("/") again unless new --skip-chdir is specified Since commit v8.22-94-g99960ee, chroot(1) skips the chroot(2) syscall for "/" arguments (and synonyms). The problem is that it also skips the following chdir("/") call in that case. The latter breaks existing scripts which expect "/" to be the working directory inside the chroot. While the first part of the change - i.e., skipping chroot("/") - is okay for consistency with systems where it might succeed for a non-root user, the second part might be malicious, e.g. cd /home/user && chroot '/' bin/foo In the "best" case, chroot(1) could not execute 'bin/foo' with ENOENT, but in the worst case, chroot(1) would execute '/home/user/bin/foo' in the case that exists - instead of '/bin/foo'. Revert that second part of the patch, i.e., perform the chdir("/) in the common case again - unless the new --skip-chdir option is specified. Restrict this new option to the case of "/" arguments. * src/chroot.c (SKIP_CHDIR): Add enum. (long_opts): Add entry for the new --skip-chdir option. (usage): Add --skip-chdir option, and while at it, move the other to options into alphabetical order. (main): Accept the above new option, allowing it only in the case when NEWROOT is the old "/". Move down the chdir() call after the if-clause to ensure it is run in any case - unless --skip-chdir is specified. Add a 'newroot' variable for the new root directory as it is used in a couple of places now. * tests/misc/chroot-fail.sh: Invert the last tests which check the working directory of the execvp()ed program when a "/"-like argument was passed: now expect it to be "/" - unless --skip-chdir is given. * doc/coreutils.texi (chroot invocation): Document the new option. Document that chroot(1) usually calls chdir("/") unless the new --skip-chdir option is specified. Sort options. * NEWS (Changes in behavior): Mention the fix. (New features): Mention the new option. * init.cfg (nonroot_has_perm_): Add chroot's new --skip-chdir option. * tests/cp/preserve-gid.sh (t1): Likewise. * tests/cp/special-bits.sh: Likewise. * tests/id/setgid.sh: Likewise. * tests/misc/truncate-owned-by-other.sh: Likewise. * tests/mv/sticky-to-xpart.sh: Likewise. * tests/rm/fail-2eperm.sh: Likewise. * tests/rm/no-give-up.sh: Likewise. * tests/touch/now-owned-by-other.sh: Likewise. Reported by Andreas Schwab in http://bugs.gnu.org/18062 2014-07-31 Pádraig Brady build: avoid building stdbuf on cygwin * configure.ac: Don't add stdbuf to the list of programs to build if EXEEXT is set, as that is not handled in configure.ac for libstdbuf.so yet (see bin_PRGRAMS handling in configure.ac). Also the LD_PRELOAD mechanism will need to be adjusted to support cygwin in any case, so avoid stdbuf completely in this case for now. Problem reported by Eric Blake. 2014-07-29 Eric Blake doc: clarify that floating point parses "inf" * doc/coreutils.texi (Floating point): Document handling of "inf", "infinity", "NaN", and so on. 2014-07-19 Paul Eggert maint: fix message translation glitches Problem reported by Sebastian Rasmussen in: http://bugs.gnu.org/18054 * gl/lib/randread.c (randread_error): Don't put multiple string literals inside _(...), as xgettext doesn't support that. * src/chroot.c (main): In diagnostics, don't bother to distinguish between setting the number of supplemental group IDs to a zero or to a nonzero value, as the underlying system call is the same either way. This also makes the string easier to translate correctly. 2014-07-18 Pádraig Brady maint: post-release administrivia * NEWS: Add header line for next release. * .prev-version: Record previous version. * cfg.mk (old_NEWS_hash): Auto-update. version 8.23 * NEWS: Record release date. See the source repo for older entries