Bump the version number to 5.1.1alpha and liblzma soname to 5.0.99.

This way people hopefully won't complain if these APIs
change and break code that used an older API.

COPYING.GPLv2

@@ -1,12 +1,12 @@
-		    GNU GENERAL PUBLIC LICENSE
-		       Version 2, June 1991
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
 
  Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
  51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
-			    Preamble
+                            Preamble
 
   The licenses for most software are designed to take away your
 freedom to share and change it.  By contrast, the GNU General Public
@@ -56,7 +56,7 @@ patent must be licensed for everyone's free use or not licensed at all.
   The precise terms and conditions for copying, distribution and
 modification follow.
 
-		    GNU GENERAL PUBLIC LICENSE
+                    GNU GENERAL PUBLIC LICENSE
    TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
 
   0. This License applies to any program or other work which contains
@@ -255,7 +255,7 @@ make exceptions for this.  Our decision will be guided by the two goals
 of preserving the free status of all derivatives of our free software and
 of promoting the sharing and reuse of software generally.
 
-			    NO WARRANTY
+                            NO WARRANTY
 
   11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
 FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
@@ -277,9 +277,9 @@ YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
 PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
 POSSIBILITY OF SUCH DAMAGES.
 
-		     END OF TERMS AND CONDITIONS
+                     END OF TERMS AND CONDITIONS
 
-	    How to Apply These Terms to Your New Programs
+            How to Apply These Terms to Your New Programs
 
   If you develop a new program, and you want it to be of the greatest
 possible use to the public, the best way to achieve this is to make it

COPYING.LGPLv2.1

@@ -1,5 +1,5 @@
-		  GNU LESSER GENERAL PUBLIC LICENSE
-		       Version 2.1, February 1999
+                  GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 2.1, February 1999
 
  Copyright (C) 1991, 1999 Free Software Foundation, Inc.
  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
@@ -10,7 +10,7 @@
  as the successor of the GNU Library Public License, version 2, hence
  the version number 2.1.]
 
-			    Preamble
+                            Preamble
 
   The licenses for most software are designed to take away your
 freedom to share and change it.  By contrast, the GNU General Public
@@ -112,7 +112,7 @@ modification follow.  Pay close attention to the difference between a
 former contains code derived from the library, whereas the latter must
 be combined with the library in order to run.
 
-		  GNU LESSER GENERAL PUBLIC LICENSE
+                  GNU LESSER GENERAL PUBLIC LICENSE
    TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
 
   0. This License Agreement applies to any software library or other
@@ -146,7 +146,7 @@ such a program is covered only if its contents constitute a work based
 on the Library (independent of the use of the Library in a tool for
 writing it).  Whether that is true depends on what the Library does
 and what the program that uses the Library does.
-  
+
   1. You may copy and distribute verbatim copies of the Library's
 complete source code as you receive it, in any medium, provided that
 you conspicuously and appropriately publish on each copy an
@@ -432,7 +432,7 @@ decision will be guided by the two goals of preserving the free status
 of all derivatives of our free software and of promoting the sharing
 and reuse of software generally.
 
-			    NO WARRANTY
+                            NO WARRANTY
 
   15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
 WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
@@ -455,7 +455,7 @@ FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
 SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
 DAMAGES.
 
-		     END OF TERMS AND CONDITIONS
+                     END OF TERMS AND CONDITIONS
 
            How to Apply These Terms to Your New Libraries
 
@@ -500,5 +500,3 @@ necessary.  Here is a sample; alter the names:
   Ty Coon, President of Vice
 
 That's all there is to it!
-
-

INSTALL

@@ -68,6 +68,10 @@ XZ Utils Installation
     the -O2 optimization flag ("make check" fails). Using -O1 should
     work.
 
+    A problem has been reported when using shared liblzma. Passing
+    --disable-shared to configure works around this. Alternatively,
+    putting "-64" to CFLAGS to build a 64-bit version might help too.
+
 
 1.2.2. MINIX 3
 

NEWS

@@ -1,6 +1,67 @@
 
-XZ Utils User-Visible Changes
-=============================
+XZ Utils Release Notes
+======================
+
+5.1.1alpha (2011-04-12)
+
+    * All fixes from 5.0.2
+
+    * liblzma fixes that will also be included in 5.0.3:
+
+        - A memory leak was fixed.
+
+        - lzma_stream_buffer_encode() no longer creates an empty .xz
+          Block if encoding an empty buffer. Such an empty Block with
+          LZMA2 data would trigger a bug in 5.0.1 and older (see the
+          first bullet point in 5.0.2 notes). When releasing 5.0.2,
+          I thought that no encoder creates this kind of files but
+          I was wrong.
+
+        - Validate function arguments better in a few functions. Most
+          importantly, specifying an unsupported integrity check to
+          lzma_stream_buffer_encode() no longer creates a corrupt .xz
+          file. Probably no application tries to do that, so this
+          shouldn't be a big problem in practice.
+
+        - Document that lzma_block_buffer_encode(),
+          lzma_easy_buffer_encode(), lzma_stream_encoder(), and
+          lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
+
+        - The return values of the _memusage() functions are now
+          documented better.
+
+    * Support for multithreaded compression was added using the simplest
+      method, which splits the input data into blocks and compresses
+      them independently. Other methods will be added in the future.
+      The current method has room for improvement, e.g. it is possible
+      to reduce the memory usage.
+
+    * Added the options --single-stream and --block-size=SIZE to xz.
+
+    * xzdiff and xzgrep now support .lzo files if lzop is installed.
+      The .tzo suffix is also recognized as a shorthand for .tar.lzo.
+
+    * Support for short 8.3 filenames under DOS was added to xz. It is
+      experimental and may change before it gets into a stable release.
+
+
+5.0.2 (2011-04-01)
+
+    * LZMA2 decompressor now correctly accepts LZMA2 streams with no
+      uncompressed data. Previously it considered them corrupt. The
+      bug can affect applications that use raw LZMA2 streams. It is
+      very unlikely to affect .xz files because no compressor creates
+      .xz files with empty LZMA2 streams. (Empty .xz files are a
+      different thing than empty LZMA2 streams.)
+
+    * "xz --suffix=.foo filename.foo" now refuses to compress the
+      file due to it already having the suffix .foo. It was already
+      documented on the man page, but the code lacked the test.
+
+    * "xzgrep -l foo bar.xz" works now.
+
+    * Polish translation was added.
+
 
 5.0.1 (2011-01-29)
 

THANKS

@@ -10,6 +10,8 @@ has been important. :-) In alphabetical order:
   - Karl Berry
   - Anders F. Björklund
   - Emmanuel Blot
+  - Martin Blumenstingl
+  - Jakub Bogusz
   - Trent W. Buck
   - David Burklund
   - Daniel Mealha Cabrita
@@ -22,6 +24,7 @@ has been important. :-) In alphabetical order:
   - Gilles Espinasse
   - Denis Excoffier
   - Mike Frysinger
+  - Jason Gorski
   - Juan Manuel Guerrero
   - Joachim Henke
   - Peter Ivanov
@@ -35,6 +38,7 @@ has been important. :-) In alphabetical order:
   - Hin-Tak Leung
   - Andraž 'ruskie' Levstik
   - Lorenzo De Liso
+  - Gregory Margo
   - Jim Meyering
   - Rafał Mużyło
   - Adrien Nader

TODO

@@ -39,7 +39,10 @@ Missing features
     xz doesn't support copying extended attributes, access control
     lists etc. from source to target file.
 
-    Multithreaded compression
+    Multithreaded compression:
+      - Reduce memory usage of the current method.
+      - Implement threaded match finders.
+      - Implement pigz-style threading in LZMA2.
 
     Multithreaded decompression
 
@@ -50,6 +53,13 @@ Missing features
     It will be a separate library that supports uncompressed, .gz,
     .bz2, .lzma, and .xz files.
 
+    Check the first 0x00 byte of LZMA data.
+
+    Support changing lzma_options_lzma.mode with lzma_filters_update().
+
+    Support LZMA_FULL_FLUSH for lzma_stream_decoder() to stop at
+    Block and Stream boundaries.
+
     lzma_strerror() to convert lzma_ret to human readable form?
     This is tricky, because the same error codes are used with
     slightly different meanings, and this cannot be fixed anymore.

configure.ac

@@ -431,11 +431,13 @@ AC_USE_SYSTEM_EXTENSIONS
 if test "x$enable_threads" = xyes; then
 	echo
 	echo "Threading support:"
-	ACX_PTHREAD
+	AX_PTHREAD
 	LIBS="$LIBS $PTHREAD_LIBS"
 	AM_CFLAGS="$AM_CFLAGS $PTHREAD_CFLAGS"
 	CC="$PTHREAD_CC"
+	AC_SEARCH_LIBS([clock_gettime], [rt])
 fi
+AM_CONDITIONAL([COND_THREADS], [test "x$ax_pthread_ok" = xyes])
 
 echo
 echo "Initializing Libtool:"
@@ -517,6 +519,9 @@ gl_GETOPT
 # Find the best function to set timestamps.
 AC_CHECK_FUNCS([futimens futimes futimesat utimes utime], [break])
 
+# This is nice to have but not mandatory.
+AC_CHECK_FUNCS([posix_fadvise])
+
 TUKLIB_PROGNAME
 TUKLIB_INTEGER
 TUKLIB_PHYSMEM

doc/lzma-file-format.txt

@@ -40,7 +40,7 @@ The .lzma File Format
 
 0.2. Changes
 
-        Last modified: 2009-05-01 11:15+0300
+        Last modified: 2011-04-12 11:55+0300
 
 
 1. File Format
@@ -161,6 +161,6 @@ The .lzma File Format
         XZ Utils - The next generation of LZMA Utils
         http://tukaani.org/xz/
 
-        The .xz file format - The successor of the the .lzma format
+        The .xz file format - The successor of the .lzma format
         http://tukaani.org/xz/xz-file-format.txt
 

dos/INSTALL.txt

@@ -1,6 +1,6 @@
 
-XZ Utils on DOS
-===============
+Building XZ Utils for DOS
+=========================
 
 Introduction
 
@@ -77,12 +77,3 @@ Building
     are not built. Having e.g. xzdec.exe doesn't save much space compared
     to xz.exe, because the DJGPP runtime makes the .exe quite big anyway.
 
-
-Bugs
-
-    xz doesn't necessarily work in Dosbox. It should work in DOSEMU.
-
-    Pressing Ctrl-c or Ctrl-Break won't remove the incomplete target file
-    when running under Windows XP Command Prompt (something goes wrong
-    with SIGINT handling). It works correctly under Windows 95/98/98SE/ME.
-

dos/README.txt

@@ -0,0 +1,123 @@
+
+XZ Utils on DOS
+===============
+
+DOS-specific filename handling
+
+    xz detects at runtime if long filename (LFN) support is
+    available and will use it by default. It can be disabled by
+    setting an environment variable:
+
+        set lfn=n
+
+    When xz is in LFN mode, it behaves pretty much the same as it
+    does on other operating systems. Examples:
+
+        xz foo.tar          -> foo.tar.xz
+        xz -d foo.tar.xz    -> foo.tar
+
+        xz -F lzma foo.tar  -> foo.tar.lzma
+        xz -d foo.tar.lzma  -> foo.tar
+
+    When LFN support isn't available or it is disabled with LFN=n
+    environment setting, xz works in short filename (SFN) mode. This
+    affects filename suffix handling when compressing.
+
+    When compressing to the .xz format in SFN mode:
+
+      - Files without an extension get .xz just like on LFN systems.
+
+      - *.tar files become *.txz (shorthand for *.tar.xz). *.txz
+        is recognized by xz on all supported operating systems.
+        (Try to avoid confusing this with gzipped .txt files.)
+
+      - Files with 1-3 character extension have their extension modified
+        so that the last character is a dash ("-"). If the extension
+        is already three characters, the last character is lost. The
+        resulting *.?- or *.??- filename is recognized in LFN mode, but
+        it isn't recognized by xz on other operating systems.
+
+    Examples:
+
+        xz foo              -> foo.xz     |   xz -d foo.xz    -> foo
+        xz foo.tar          -> foo.txz    |   xz -d foo.txz   -> foo.tar
+        xz foo.c            -> foo.c-     |   xz -d foo.c-    -> foo.c
+        xz read.me          -> read.me-   |   xz -d read.me-  -> read.me
+        xz foo.txt          -> foo.tx-    |   xz -d foo.tx-   -> foo.tx   !
+
+    Note that in the last example above, the third character of the
+    filename extension is lost.
+
+    When compressing to the legacy .lzma format in SFN mode:
+
+      - *.tar files become *.tlz (shorthand for *.tar.lzma). *.tlz
+        is recognized by xz on all supported operating systems.
+
+      - Other files become *.lzm. The original filename extension
+        is lost. *.lzm is recognized also in LFN mode, but it is not
+        recognized by xz on other operating systems.
+
+    Examples:
+
+        xz -F lzma foo      -> foo.lzm    |   xz -d foo.lzm   -> foo
+        xz -F lzma foo.tar  -> foo.tlz    |   xz -d foo.tlz   -> foo.tar
+        xz -F lzma foo.c    -> foo.lzm    |   xz -d foo.lzm   -> foo      !
+        xz -F lzma read.me  -> read.lzm   |   xz -d read.lzm  -> read     !
+        xz -F lzma foo.txt  -> foo.lzm    |   xz -d foo.lzm   -> foo      !
+
+    When compressing with a custom suffix (-S .SUF, --suffix=.SUF) to
+    any file format:
+
+      - If the suffix begins with a dot, the filename extension is
+        replaced with the new suffix. The original extension is lost.
+
+      - If the suffix doesn't begin with a dot and the filename has no
+        extension and the filename given on the command line doesn't
+        have a dot at the end, the custom suffix is appended just like
+        on LFN systems.
+
+      - If the suffix doesn't begin with a dot and the filename has
+        an extension (or an extension-less filename is given with a dot
+        at the end), the last 1-3 characters of the filename extension
+        may get overwritten to fit the given custom suffix.
+
+    Examples:
+
+        xz -S x foo        -> foox      |  xz -dS x foox      -> foo
+        xz -S x foo.       -> foo.x     |  xz -dS x foo.x     -> foo
+        xz -S .x foo       -> foo.x     |  xz -dS .x foo.x    -> foo
+        xz -S .x foo.      -> foo.x     |  xz -dS .x foo.x    -> foo
+        xz -S x.y foo      -> foox.y    |  xz -dS x.y foox.y  -> foo
+        xz -S .a foo.c     -> foo.a     |  xz -dS .a foo.a    -> foo      !
+        xz -S a  foo.c     -> foo.ca    |  xz -dS a foo.ca    -> foo.c
+        xz -S ab foo.c     -> foo.cab   |  xz -dS ab foo.cab  -> foo.c
+        xz -S ab read.me   -> read.mab  |  xz -dS ab read.mab -> read.m   !
+        xz -S ab foo.txt   -> foo.tab   |  xz -dS ab foo.tab  -> foo.t    !
+        xz -S abc foo.txt  -> foo.abc   |  xz -dS abc foo.abc -> foo      !
+
+    When decompressing, the suffix handling in SFN mode is the same as
+    in LFN mode. The DOS-specific filenames *.lzm, *.?-, and *.??- are
+    recognized also in LFN mode.
+
+    xz handles certain uncommon situations safely:
+
+      - If the generated output filename refers to the same file as
+        the input file, xz detects this and refuses to compress or
+        decompress the input file even if --force is used. This can
+        happen when giving an overlong filename in SFN mode. E.g.
+        "xz -S x foo.texinfo" would try to write to foo.tex which on
+        SFN system is the same file as foo.texinfo.
+
+      - If the generated output filename is a special file like "con"
+        or "prn", xz detects this and refuses to compress or decompress
+        the input file even if --force is used.
+
+
+Bugs
+
+    xz doesn't necessarily work in Dosbox. It should work in DOSEMU.
+
+    Pressing Ctrl-c or Ctrl-Break won't remove the incomplete target file
+    when running under Windows XP Command Prompt (something goes wrong
+    with SIGINT handling). It works correctly under Windows 95/98/98SE/ME.
+

m4/ax_pthread.m4

@@ -1,93 +1,88 @@
-##### http://autoconf-archive.cryp.to/acx_pthread.html
+# ===========================================================================
+#        http://www.gnu.org/software/autoconf-archive/ax_pthread.html
+# ===========================================================================
 #
 # SYNOPSIS
 #
-#   ACX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]])
+#   AX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]])
 #
 # DESCRIPTION
 #
-#   This macro figures out how to build C programs using POSIX threads.
-#   It sets the PTHREAD_LIBS output variable to the threads library and
-#   linker flags, and the PTHREAD_CFLAGS output variable to any special
-#   C compiler flags that are needed. (The user can also force certain
-#   compiler flags/libs to be tested by setting these environment
-#   variables.)
+#   This macro figures out how to build C programs using POSIX threads. It
+#   sets the PTHREAD_LIBS output variable to the threads library and linker
+#   flags, and the PTHREAD_CFLAGS output variable to any special C compiler
+#   flags that are needed. (The user can also force certain compiler
+#   flags/libs to be tested by setting these environment variables.)
 #
 #   Also sets PTHREAD_CC to any special C compiler that is needed for
-#   multi-threaded programs (defaults to the value of CC otherwise).
-#   (This is necessary on AIX to use the special cc_r compiler alias.)
+#   multi-threaded programs (defaults to the value of CC otherwise). (This
+#   is necessary on AIX to use the special cc_r compiler alias.)
 #
-#   NOTE: You are assumed to not only compile your program with these
-#   flags, but also link it with them as well. e.g. you should link
-#   with $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS
-#   $LIBS
+#   NOTE: You are assumed to not only compile your program with these flags,
+#   but also link it with them as well. e.g. you should link with
+#   $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
 #
-#   If you are only building threads programs, you may wish to use
-#   these variables in your default LIBS, CFLAGS, and CC:
+#   If you are only building threads programs, you may wish to use these
+#   variables in your default LIBS, CFLAGS, and CC:
 #
-#          LIBS="$PTHREAD_LIBS $LIBS"
-#          CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
-#          CC="$PTHREAD_CC"
+#     LIBS="$PTHREAD_LIBS $LIBS"
+#     CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+#     CC="$PTHREAD_CC"
 #
-#   In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute
-#   constant has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to
-#   that name (e.g. PTHREAD_CREATE_UNDETACHED on AIX).
+#   In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant
+#   has a nonstandard name, defines PTHREAD_CREATE_JOINABLE to that name
+#   (e.g. PTHREAD_CREATE_UNDETACHED on AIX).
 #
-#   ACTION-IF-FOUND is a list of shell commands to run if a threads
-#   library is found, and ACTION-IF-NOT-FOUND is a list of commands to
-#   run it if it is not found. If ACTION-IF-FOUND is not specified, the
-#   default action will define HAVE_PTHREAD.
+#   ACTION-IF-FOUND is a list of shell commands to run if a threads library
+#   is found, and ACTION-IF-NOT-FOUND is a list of commands to run it if it
+#   is not found. If ACTION-IF-FOUND is not specified, the default action
+#   will define HAVE_PTHREAD.
 #
-#   Please let the authors know if this macro fails on any platform, or
-#   if you have any other suggestions or comments. This macro was based
-#   on work by SGJ on autoconf scripts for FFTW (http://www.fftw.org/)
-#   (with help from M. Frigo), as well as ac_pthread and hb_pthread
-#   macros posted by Alejandro Forero Cuervo to the autoconf macro
-#   repository. We are also grateful for the helpful feedback of
-#   numerous users.
+#   Please let the authors know if this macro fails on any platform, or if
+#   you have any other suggestions or comments. This macro was based on work
+#   by SGJ on autoconf scripts for FFTW (http://www.fftw.org/) (with help
+#   from M. Frigo), as well as ac_pthread and hb_pthread macros posted by
+#   Alejandro Forero Cuervo to the autoconf macro repository. We are also
+#   grateful for the helpful feedback of numerous users.
 #
-# LAST MODIFICATION
+# LICENSE
 #
-#   2007-07-29
+#   Copyright (c) 2008 Steven G. Johnson <stevenj@alum.mit.edu>
 #
-# COPYLEFT
-#
-#   Copyright (c) 2007 Steven G. Johnson <stevenj@alum.mit.edu>
-#
-#   This program is free software: you can redistribute it and/or
-#   modify it under the terms of the GNU General Public License as
-#   published by the Free Software Foundation, either version 3 of the
-#   License, or (at your option) any later version.
+#   This program is free software: you can redistribute it and/or modify it
+#   under the terms of the GNU General Public License as published by the
+#   Free Software Foundation, either version 3 of the License, or (at your
+#   option) any later version.
 #
 #   This program is distributed in the hope that it will be useful, but
 #   WITHOUT ANY WARRANTY; without even the implied warranty of
-#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-#   General Public License for more details.
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+#   Public License for more details.
 #
-#   You should have received a copy of the GNU General Public License
-#   along with this program. If not, see
-#   <http://www.gnu.org/licenses/>.
+#   You should have received a copy of the GNU General Public License along
+#   with this program. If not, see <http://www.gnu.org/licenses/>.
 #
-#   As a special exception, the respective Autoconf Macro's copyright
-#   owner gives unlimited permission to copy, distribute and modify the
-#   configure scripts that are the output of Autoconf when processing
-#   the Macro. You need not follow the terms of the GNU General Public
-#   License when using or distributing such scripts, even though
-#   portions of the text of the Macro appear in them. The GNU General
-#   Public License (GPL) does govern all other use of the material that
-#   constitutes the Autoconf Macro.
+#   As a special exception, the respective Autoconf Macro's copyright owner
+#   gives unlimited permission to copy, distribute and modify the configure
+#   scripts that are the output of Autoconf when processing the Macro. You
+#   need not follow the terms of the GNU General Public License when using
+#   or distributing such scripts, even though portions of the text of the
+#   Macro appear in them. The GNU General Public License (GPL) does govern
+#   all other use of the material that constitutes the Autoconf Macro.
 #
-#   This special exception to the GPL applies to versions of the
-#   Autoconf Macro released by the Autoconf Macro Archive. When you
-#   make and distribute a modified version of the Autoconf Macro, you
-#   may extend this special exception to the GPL to apply to your
-#   modified version as well.
+#   This special exception to the GPL applies to versions of the Autoconf
+#   Macro released by the Autoconf Archive. When you make and distribute a
+#   modified version of the Autoconf Macro, you may extend this special
+#   exception to the GPL to apply to your modified version as well.
 
-AC_DEFUN([ACX_PTHREAD], [
+#serial 11
+
+AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
+AC_DEFUN([AX_PTHREAD], [
 AC_REQUIRE([AC_CANONICAL_HOST])
 AC_LANG_SAVE
 AC_LANG_C
-acx_pthread_ok=no
+ax_pthread_ok=no
 
 # We used to check for pthread.h first, but this fails if pthread.h
 # requires special compiler flags (e.g. on True64 or Sequent).
@@ -102,9 +97,9 @@ if test x"$PTHREAD_LIBS$PTHREAD_CFLAGS" != x; then
         save_LIBS="$LIBS"
         LIBS="$PTHREAD_LIBS $LIBS"
         AC_MSG_CHECKING([for pthread_join in LIBS=$PTHREAD_LIBS with CFLAGS=$PTHREAD_CFLAGS])
-        AC_TRY_LINK_FUNC(pthread_join, acx_pthread_ok=yes)
-        AC_MSG_RESULT($acx_pthread_ok)
-        if test x"$acx_pthread_ok" = xno; then
+        AC_TRY_LINK_FUNC(pthread_join, ax_pthread_ok=yes)
+        AC_MSG_RESULT($ax_pthread_ok)
+        if test x"$ax_pthread_ok" = xno; then
                 PTHREAD_LIBS=""
                 PTHREAD_CFLAGS=""
         fi
@@ -122,7 +117,7 @@ fi
 # which indicates that we try without any flags at all, and "pthread-config"
 # which is a program returning the flags for the Pth emulation library.
 
-acx_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
+ax_pthread_flags="pthreads none -Kthread -kthread lthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
 
 # The ordering *is* (sometimes) important.  Some notes on the
 # individual items follow:
@@ -155,12 +150,16 @@ case "${host_cpu}-${host_os}" in
         # who knows whether they'll stub that too in a future libc.)  So,
         # we'll just look for -pthreads and -lpthread first:
 
-        acx_pthread_flags="-pthreads pthread -mt -pthread $acx_pthread_flags"
+        ax_pthread_flags="-pthreads pthread -mt -pthread $ax_pthread_flags"
         ;;
+
+	*-darwin*)
+	ax_pthread_flags="-pthread $ax_pthread_flags"
+	;;
 esac
 
-if test x"$acx_pthread_ok" = xno; then
-for flag in $acx_pthread_flags; do
+if test x"$ax_pthread_ok" = xno; then
+for flag in $ax_pthread_flags; do
 
         case $flag in
                 none)
@@ -173,8 +172,8 @@ for flag in $acx_pthread_flags; do
                 ;;
 
 		pthread-config)
-		AC_CHECK_PROG(acx_pthread_config, pthread-config, yes, no)
-		if test x"$acx_pthread_config" = xno; then continue; fi
+		AC_CHECK_PROG(ax_pthread_config, pthread-config, yes, no)
+		if test x"$ax_pthread_config" = xno; then continue; fi
 		PTHREAD_CFLAGS="`pthread-config --cflags`"
 		PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`"
 		;;
@@ -199,17 +198,22 @@ for flag in $acx_pthread_flags; do
         # pthread_cleanup_push because it is one of the few pthread
         # functions on Solaris that doesn't have a non-functional libc stub.
         # We try pthread_create on general principles.
-        AC_TRY_LINK([#include <pthread.h>],
-                    [pthread_t th; pthread_join(th, 0);
-                     pthread_attr_init(0); pthread_cleanup_push(0, 0);
-                     pthread_create(0,0,0,0); pthread_cleanup_pop(0); ],
-                    [acx_pthread_ok=yes])
+        AC_TRY_LINK([#include <pthread.h>
+		     static void routine(void* a) {a=0;}
+		     static void* start_routine(void* a) {return a;}],
+                    [pthread_t th; pthread_attr_t attr;
+                     pthread_create(&th,0,start_routine,0);
+                     pthread_join(th, 0);
+                     pthread_attr_init(&attr);
+                     pthread_cleanup_push(routine, 0);
+                     pthread_cleanup_pop(0); ],
+                    [ax_pthread_ok=yes])
 
         LIBS="$save_LIBS"
         CFLAGS="$save_CFLAGS"
 
-        AC_MSG_RESULT($acx_pthread_ok)
-        if test "x$acx_pthread_ok" = xyes; then
+        AC_MSG_RESULT($ax_pthread_ok)
+        if test "x$ax_pthread_ok" = xyes; then
                 break;
         fi
 
@@ -219,7 +223,7 @@ done
 fi
 
 # Various other checks:
-if test "x$acx_pthread_ok" = xyes; then
+if test "x$ax_pthread_ok" = xyes; then
         save_LIBS="$LIBS"
         LIBS="$PTHREAD_LIBS $LIBS"
         save_CFLAGS="$CFLAGS"
@@ -268,12 +272,12 @@ AC_SUBST(PTHREAD_CFLAGS)
 AC_SUBST(PTHREAD_CC)
 
 # Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND:
-if test x"$acx_pthread_ok" = xyes; then
+if test x"$ax_pthread_ok" = xyes; then
         ifelse([$1],,AC_DEFINE(HAVE_PTHREAD,1,[Define if you have POSIX threads libraries and header files.]),[$1])
         :
 else
-        acx_pthread_ok=no
+        ax_pthread_ok=no
         $2
 fi
 AC_LANG_RESTORE
-])dnl ACX_PTHREAD
+])dnl AX_PTHREAD

macosx/build.sh

@@ -0,0 +1,92 @@
+#!/bin/sh
+
+###############################################################################
+# Author: Anders F Björklund <afb@users.sourceforge.net>
+#
+# This file has been put into the public domain.
+# You can do whatever you want with this file.
+###############################################################################
+
+mkdir -p Root
+mkdir -p Resources
+
+# Abort immediately if something goes wrong.
+set -e
+
+# Clean up if it was already configured.
+[ -f Makefile ] && make distclean
+
+# Build the regular fat program
+
+CC="gcc-4.0" \
+CFLAGS="-O2 -g -arch ppc -arch ppc64 -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4" \
+../configure --disable-dependency-tracking --disable-xzdec --disable-lzmadec i686-apple-darwin8
+
+make
+
+make check
+
+make DESTDIR=`pwd`/Root install
+
+make distclean
+
+# Build the size-optimized program
+
+CC="gcc-4.0" \
+CFLAGS="-Os -g -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4" \
+../configure --disable-dependency-tracking --disable-shared --disable-nls --disable-encoders --enable-small --disable-threads i686-apple-darwin8
+
+make -C src/liblzma
+make -C src/xzdec
+make -C src/xzdec DESTDIR=`pwd`/Root install
+
+cp -a ../extra Root/usr/local/share/doc/xz
+
+make distclean
+
+# Strip debugging symbols and make relocatable
+
+for bin in xz lzmainfo xzdec lzmadec; do
+    strip -S Root/usr/local/bin/$bin
+    install_name_tool -change /usr/local/lib/liblzma.5.dylib @executable_path/../lib/liblzma.5.dylib Root/usr/local/bin/$bin
+done
+
+for lib in liblzma.5.dylib; do
+    strip -S Root/usr/local/lib/$lib
+    install_name_tool -id @executable_path/../lib/liblzma.5.dylib Root/usr/local/lib/$lib
+done
+
+strip -S  Root/usr/local/lib/liblzma.a
+rm -f Root/usr/local/lib/liblzma.la
+
+# Include pkg-config while making relocatable
+
+sed -e 's|prefix=/usr/local|prefix=${pcfiledir}/../..|' < Root/usr/local/lib/pkgconfig/liblzma.pc > Root/liblzma.pc
+mv Root/liblzma.pc Root/usr/local/lib/pkgconfig/liblzma.pc
+
+# Create tarball, but without the HFS+ attrib
+
+rmdir debug lib po src/liblzma/api src/liblzma src/lzmainfo src/scripts src/xz src/xzdec src tests
+
+( cd Root/usr/local; COPY_EXTENDED_ATTRIBUTES_DISABLE=true COPYFILE_DISABLE=true tar cvjf ../../../XZ.tbz * )
+
+# Include documentation files for package
+
+cp -p ../README Resources/ReadMe.txt
+cp -p ../COPYING Resources/License.txt
+
+# Make an Installer.app package
+
+ID="org.tukaani.xz"
+VERSION=`cd ..; sh build-aux/version.sh`
+PACKAGEMAKER=/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker
+$PACKAGEMAKER -r Root/usr/local -l /usr/local -e Resources -i $ID -n $VERSION -t XZ -o XZ.pkg -g 10.4 --verbose
+
+# Put the package in a disk image
+
+hdiutil create -fs HFS+ -format UDZO -quiet -srcfolder XZ.pkg -ov XZ.dmg
+hdiutil internet-enable -yes -quiet XZ.dmg
+
+echo
+echo "Build completed successfully."
+echo

po/LINGUAS

@@ -1,3 +1,4 @@
 cs
 de
 it
+pl

po/pl.po

@@ -0,0 +1,825 @@
+# Polish translation for xz.
+# This file is in the public domain.
+# Jakub Bogusz <qboosh@pld-linux.org>, 2011.
+#
+msgid ""
+msgstr ""
+"Project-Id-Version: xz 5.0.1\n"
+"Report-Msgid-Bugs-To: lasse.collin@tukaani.org\n"
+"POT-Creation-Date: 2011-01-28 20:01+0200\n"
+"PO-Revision-Date: 2011-02-02 16:51+0100\n"
+"Last-Translator: Jakub Bogusz <qboosh@pld-linux.org>\n"
+"Language-Team: Polish <translation-team-pl@lists.sourceforge.net>\n"
+"Language: pl\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Plural-Forms: nplurals=3; plural=n==1 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;\n"
+
+#: src/xz/args.c:333
+#, c-format
+msgid "%s: Unknown file format type"
+msgstr "%s: Nieznany typ formatu pliku"
+
+#: src/xz/args.c:356 src/xz/args.c:364
+#, c-format
+msgid "%s: Unsupported integrity check type"
+msgstr "%s: Nieobsługiwany typ kontroli spójności"
+
+#: src/xz/args.c:382
+msgid "Only one file can be specified with `--files' or `--files0'."
+msgstr "Wraz z opcją `--files' lub `--files0' można podać tylko jeden plik."
+
+#: src/xz/args.c:445
+#, c-format
+msgid "The environment variable %s contains too many arguments"
+msgstr "Zmienna środowiskowa %s zawiera zbyt dużo argumentów"
+
+#: src/xz/coder.c:95
+msgid "Maximum number of filters is four"
+msgstr "Maksymalna liczba filtrów to cztery"
+
+#: src/xz/coder.c:108
+msgid "Memory usage limit is too low for the given filter setup."
+msgstr "Limit użycia pamięci jest zbyt mały dla podanej konfiguracji filtra."
+
+#: src/xz/coder.c:129
+msgid "Using a preset in raw mode is discouraged."
+msgstr "Użycie ustawień predefiniowanych w trybie surowym jest odradzane."
+
+#: src/xz/coder.c:131
+msgid "The exact options of the presets may vary between software versions."
+msgstr "Dokładne opcje ustawień predefiniowanych mogą różnić się między wersjami oprogramowania."
+
+#: src/xz/coder.c:157
+msgid "The .lzma format supports only the LZMA1 filter"
+msgstr "Format .lzma obsługuje tylko filtr LZMA1"
+
+#: src/xz/coder.c:165
+msgid "LZMA1 cannot be used with the .xz format"
+msgstr "LZMA1 nie może być używany z formatem .xz"
+
+#: src/xz/coder.c:182
+msgid "Unsupported filter chain or filter options"
+msgstr "Nieobsługiwany łańcuch filtrów lub opcje filtra"
+
+#: src/xz/coder.c:190
+#, c-format
+msgid "Decompression will need %s MiB of memory."
+msgstr "Dekompresja będzie wymagała %s MiB pamięci."
+
+#: src/xz/coder.c:247
+#, c-format
+msgid "Adjusted LZMA%c dictionary size from %s MiB to %s MiB to not exceed the memory usage limit of %s MiB"
+msgstr "Skorygowano rozmiar słownika LZMA%c z %s MiB do %s MiB aby nie przekroczyć limitu użycia pamięci %s MiB"
+
+#. TRANSLATORS: When compression or decompression finishes,
+#. and xz is going to remove the source file, xz first checks
+#. if the source file still exists, and if it does, does its
+#. device and inode numbers match what xz saw when it opened
+#. the source file. If these checks fail, this message is
+#. shown, %s being the filename, and the file is not deleted.
+#. The check for device and inode numbers is there, because
+#. it is possible that the user has put a new file in place
+#. of the original file, and in that case it obviously
+#. shouldn't be removed.
+#: src/xz/file_io.c:137
+#, c-format
+msgid "%s: File seems to have been moved, not removing"
+msgstr "%s: Plik wygląda na przeniesiony, nie zostanie usunięty"
+
+#: src/xz/file_io.c:144 src/xz/file_io.c:589
+#, c-format
+msgid "%s: Cannot remove: %s"
+msgstr "%s: Nie można usunąć: %s"
+
+#: src/xz/file_io.c:169
+#, c-format
+msgid "%s: Cannot set the file owner: %s"
+msgstr "%s: Nie można ustawić właściciela pliku: %s"
+
+#: src/xz/file_io.c:175
+#, c-format
+msgid "%s: Cannot set the file group: %s"
+msgstr "%s: Nie można ustawić grupy pliku: %s"
+
+#: src/xz/file_io.c:194
+#, c-format
+msgid "%s: Cannot set the file permissions: %s"
+msgstr "%s: Nie można ustawić uprawnień pliku: %s"
+
+#: src/xz/file_io.c:337 src/xz/file_io.c:420
+#, c-format
+msgid "%s: Is a symbolic link, skipping"
+msgstr "%s: Jest dowiązaniem symbolicznym, pominięto"
+
+#: src/xz/file_io.c:455
+#, c-format
+msgid "%s: Is a directory, skipping"
+msgstr "%s: Jest katalogiem, pominięto"
+
+#: src/xz/file_io.c:461
+#, c-format
+msgid "%s: Not a regular file, skipping"
+msgstr "%s: Nie jest zwykłym plikiem, pominięto"
+
+#: src/xz/file_io.c:478
+#, c-format
+msgid "%s: File has setuid or setgid bit set, skipping"
+msgstr "%s: Plik ma ustawiony bit setuid lub setgid, pominięto"
+
+#: src/xz/file_io.c:485
+#, c-format
+msgid "%s: File has sticky bit set, skipping"
+msgstr "%s: Plik ma ustawiony bit sticky, pominięto"
+
+#: src/xz/file_io.c:492
+#, c-format
+msgid "%s: Input file has more than one hard link, skipping"
+msgstr "%s: Plik wejściowy ma więcej niż jedno dowiązanie zwykłe, pominięto"
+
+#: src/xz/file_io.c:713
+#, c-format
+msgid "Error restoring the O_APPEND flag to standard output: %s"
+msgstr "Błąd podczas odtwarzania flagi O_APPEND dla standardowego wyjścia: %s"
+
+#: src/xz/file_io.c:725
+#, c-format
+msgid "%s: Closing the file failed: %s"
+msgstr "%s: Zamknięcie pliku nie powiodło się: %s"
+
+#: src/xz/file_io.c:761 src/xz/file_io.c:945
+#, c-format
+msgid "%s: Seeking failed when trying to create a sparse file: %s"
+msgstr "%s: Zmiana pozycji nie powiodła się podczas próby utworzenia pliku rzadkiego: %s"
+
+#: src/xz/file_io.c:820
+#, c-format
+msgid "%s: Read error: %s"
+msgstr "%s: Błąd odczytu: %s"
+
+#: src/xz/file_io.c:843
+#, c-format
+msgid "%s: Error seeking the file: %s"
+msgstr "%s: Błąd podczas zmiany pozycji w pliku: %s"
+
+#: src/xz/file_io.c:853
+#, c-format
+msgid "%s: Unexpected end of file"
+msgstr "%s: Nieoczekiwany koniec pliku"
+
+#: src/xz/file_io.c:903
+#, c-format
+msgid "%s: Write error: %s"
+msgstr "%s: Błąd zapisu: %s"
+
+#: src/xz/hardware.c:100
+msgid "Disabled"
+msgstr "Wyłączony"
+
+#. TRANSLATORS: Test with "xz --info-memory" to see if
+#. the alignment looks nice.
+#: src/xz/hardware.c:119
+msgid "Total amount of physical memory (RAM): "
+msgstr "Całkowita ilość pamięci fizycznej (RAM): "
+
+#: src/xz/hardware.c:121
+msgid "Memory usage limit for compression:    "
+msgstr "Limit użycia pamięci dla kompresji:      "
+
+#: src/xz/hardware.c:123
+msgid "Memory usage limit for decompression:  "
+msgstr "Limit użycia pamięci dla dekompresji:    "
+
+#. TRANSLATORS: Indicates that there is no integrity check.
+#. This string is used in tables, so the width must not
+#. exceed ten columns with a fixed-width font.
+#: src/xz/list.c:62
+msgid "None"
+msgstr "Brak"
+
+#. TRANSLATORS: Indicates that integrity check name is not known,
+#. but the Check ID is known (here 2). This and other "Unknown-N"
+#. strings are used in tables, so the width must not exceed ten
+#. columns with a fixed-width font. It's OK to omit the dash if
+#. you need space for one extra letter, but don't use spaces.
+#: src/xz/list.c:69
+msgid "Unknown-2"
+msgstr "Nieznany-2"
+
+#: src/xz/list.c:70
+msgid "Unknown-3"
+msgstr "Nieznany-3"
+
+#: src/xz/list.c:72
+msgid "Unknown-5"
+msgstr "Nieznany-5"
+
+#: src/xz/list.c:73
+msgid "Unknown-6"
+msgstr "Nieznany-6"
+
+#: src/xz/list.c:74
+msgid "Unknown-7"
+msgstr "Nieznany-7"
+
+#: src/xz/list.c:75
+msgid "Unknown-8"
+msgstr "Nieznany-8"
+
+#: src/xz/list.c:76
+msgid "Unknown-9"
+msgstr "Nieznany-9"
+
+#: src/xz/list.c:78
+msgid "Unknown-11"
+msgstr "Nieznany11"
+
+#: src/xz/list.c:79
+msgid "Unknown-12"
+msgstr "Nieznany12"
+
+#: src/xz/list.c:80
+msgid "Unknown-13"
+msgstr "Nieznany13"
+
+#: src/xz/list.c:81
+msgid "Unknown-14"
+msgstr "Nieznany14"
+
+#: src/xz/list.c:82
+msgid "Unknown-15"
+msgstr "Nieznany15"
+
+#: src/xz/list.c:126
+#, c-format
+msgid "%s: File is empty"
+msgstr "%s: Plik jest pusty"
+
+#: src/xz/list.c:131
+#, c-format
+msgid "%s: Too small to be a valid .xz file"
+msgstr "%s: Za mały na poprawny plik .xz"
+
+#. TRANSLATORS: These are column headings. From Strms (Streams)
+#. to Ratio, the columns are right aligned. Check and Filename
+#. are left aligned. If you need longer words, it's OK to
+#. use two lines here. Test with "xz -l foo.xz".
+#: src/xz/list.c:612
+msgid "Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename"
+msgstr "Strum.  Bloki    Spakowany  Rozpakowany  Wsp.  Kontrola Nazwa pliku"
+
+#: src/xz/list.c:652
+#, c-format
+msgid "  Streams:            %s\n"
+msgstr "  Strumienie:            %s\n"
+
+#: src/xz/list.c:654
+#, c-format
+msgid "  Blocks:             %s\n"
+msgstr "  Bloki:                 %s\n"
+
+#: src/xz/list.c:656
+#, c-format
+msgid "  Compressed size:    %s\n"
+msgstr "  Rozmiar spakowany:     %s\n"
+
+#: src/xz/list.c:659
+#, c-format
+msgid "  Uncompressed size:  %s\n"
+msgstr "  Rozmiar rozpakowany:   %s\n"
+
+#: src/xz/list.c:662
+#, c-format
+msgid "  Ratio:              %s\n"
+msgstr "  Współczynnik:          %s\n"
+
+#: src/xz/list.c:664
+#, c-format
+msgid "  Check:              %s\n"
+msgstr "  Kontrola spójności:    %s\n"
+
+#: src/xz/list.c:665
+#, c-format
+msgid "  Stream padding:     %s\n"
+msgstr "  Wyrównanie strumienia: %s\n"
+
+#. TRANSLATORS: The second line is column headings. All except
+#. Check are right aligned; Check is left aligned. Test with
+#. "xz -lv foo.xz".
+#: src/xz/list.c:693
+msgid ""
+"  Streams:\n"
+"    Stream    Blocks      CompOffset    UncompOffset        CompSize      UncompSize  Ratio  Check      Padding"
+msgstr ""
+"  Strumienie:\n"
+"  Strumień     Bloki    Offset spak.    Offset rozp.        Rozm.spak.    Rozm.rozp.  Wsp.   Kontrola   Wyrównanie"
+
+#. TRANSLATORS: The second line is column headings. All
+#. except Check are right aligned; Check is left aligned.
+#: src/xz/list.c:748
+#, c-format
+msgid ""
+"  Blocks:\n"
+"    Stream     Block      CompOffset    UncompOffset       TotalSize      UncompSize  Ratio  Check"
+msgstr ""
+"  Bloki:\n"
+"  Strumień      Blok    Offset spak.    Offset rozp.  Rozm.całkowity      Rozm.rozp.  Wsp.   Kontrola"
+
+#. TRANSLATORS: These are additional column headings
+#. for the most verbose listing mode. CheckVal
+#. (Check value), Flags, and Filters are left aligned.
+#. Header (Block Header Size), CompSize, and MemUsage
+#. are right aligned. %*s is replaced with 0-120
+#. spaces to make the CheckVal column wide enough.
+#. Test with "xz -lvv foo.xz".
+#: src/xz/list.c:760
+#, c-format
+msgid "      CheckVal %*s Header  Flags        CompSize    MemUsage  Filters"
+msgstr "   S.kontr. %*sNagłówek Flagi     Rozm. spak.  Uż.pamięci  Filtry"
+
+#: src/xz/list.c:838 src/xz/list.c:1007
+#, c-format
+msgid "  Memory needed:      %s MiB\n"
+msgstr "  Wymagana pamięć:       %s MiB\n"
+
+#: src/xz/list.c:840 src/xz/list.c:1009
+#, c-format
+msgid "  Sizes in headers:   %s\n"
+msgstr "  Rozmiar w nagłówkach:  %s\n"
+
+#: src/xz/list.c:841 src/xz/list.c:1010
+msgid "Yes"
+msgstr "Tak"
+
+#: src/xz/list.c:841 src/xz/list.c:1010
+msgid "No"
+msgstr "Nie"
+
+#. TRANSLATORS: %s is an integer. Only the plural form of this
+#. message is used (e.g. "2 files"). Test with "xz -l foo.xz bar.xz".
+#: src/xz/list.c:986
+#, c-format
+msgid "%s file\n"
+msgid_plural "%s files\n"
+msgstr[0] "%s plik\n"
+msgstr[1] "%s pliki\n"
+msgstr[2] "%s plików\n"
+
+#: src/xz/list.c:999
+msgid "Totals:"
+msgstr "Sumarycznie:"
+
+#: src/xz/list.c:1000
+#, c-format
+msgid "  Number of files:    %s\n"
+msgstr "  Liczba plików:         %s\n"
+
+#: src/xz/list.c:1072
+msgid "--list works only on .xz files (--format=xz or --format=auto)"
+msgstr "--list działa tylko z plikami .xz (--format=xz lub --format=auto)"
+
+#: src/xz/list.c:1078
+msgid "--list does not support reading from standard input"
+msgstr "--list nie obsługuje odczytu ze standardowego wejścia"
+
+#: src/xz/main.c:89
+#, c-format
+msgid "%s: Error reading filenames: %s"
+msgstr "%s: Błąd odczytu nazw plików: %s"
+
+#: src/xz/main.c:96
+#, c-format
+msgid "%s: Unexpected end of input when reading filenames"
+msgstr "%s: Nieoczekiwany koniec wejścia podczas odczytu nazw plików"
+
+#: src/xz/main.c:120
+#, c-format
+msgid "%s: Null character found when reading filenames; maybe you meant to use `--files0' instead of `--files'?"
+msgstr "%s: Napotkano znak NUL podczas odczytu nazw plików; może miało być `--files0' zamiast `--files'?"
+
+#: src/xz/main.c:174
+msgid "Compression and decompression with --robot are not supported yet."
+msgstr "Kompresja i dekompresja z opcją --robot nie jest jeszcze obsługiwana."
+
+#: src/xz/main.c:231
+msgid "Cannot read data from standard input when reading filenames from standard input"
+msgstr "Nie można odczytać danych ze standardowego wejścia przy czytaniu nazw plików ze standardowego wejścia"
+
+#: src/xz/message.c:792 src/xz/message.c:842
+msgid "Internal error (bug)"
+msgstr "Błąd wewnętrzny"
+
+#: src/xz/message.c:799
+msgid "Cannot establish signal handlers"
+msgstr "Nie można ustawić obsługi sygnałów"
+
+#: src/xz/message.c:808
+msgid "No integrity check; not verifying file integrity"
+msgstr "Brak kontroli spójności; poprawność plików nie będzie weryfikowana"
+
+#: src/xz/message.c:811
+msgid "Unsupported type of integrity check; not verifying file integrity"
+msgstr "Nieobsługiwany typ kontroli spójności; poprawność plików nie będzie weryfikowana"
+
+#: src/xz/message.c:818
+msgid "Memory usage limit reached"
+msgstr "Osiągnięto limit użycia pamięci"
+
+#: src/xz/message.c:821
+msgid "File format not recognized"
+msgstr "Nie rozpoznany format pliku"
+
+#: src/xz/message.c:824
+msgid "Unsupported options"
+msgstr "Nieobsługiwane opcje"
+
+#: src/xz/message.c:827
+msgid "Compressed data is corrupt"
+msgstr "Dane skompresowane są uszkodzone"
+
+#: src/xz/message.c:830
+msgid "Unexpected end of input"
+msgstr "Nieoczekiwany koniec wejścia"
+
+#: src/xz/message.c:881
+#, c-format
+msgid "%s MiB of memory is required. The limit is %s."
+msgstr "Wymagane jest %s MiB pamięci. Limit to %s."
+
+#: src/xz/message.c:1048
+#, c-format
+msgid "%s: Filter chain: %s\n"
+msgstr "%s: Łańcuch filtrów: %s\n"
+
+#: src/xz/message.c:1058
+#, c-format
+msgid "Try `%s --help' for more information."
+msgstr "Polecenie `%s --help' pokaże więcej informacji."
+
+#: src/xz/message.c:1084
+#, c-format
+msgid ""
+"Usage: %s [OPTION]... [FILE]...\n"
+"Compress or decompress FILEs in the .xz format.\n"
+"\n"
+msgstr ""
+"Składnia: %s [OPCJA]... [PLIK]...\n"
+"Kompresja lub dekompresja PLIKÓW w formacie .xz.\n"
+"\n"
+
+#: src/xz/message.c:1091
+msgid "Mandatory arguments to long options are mandatory for short options too.\n"
+msgstr ""
+"Argumenty obowiązkowe dla opcji długich są obowiązkowe również dla opcji\n"
+"krótkich.\n"
+
+#: src/xz/message.c:1095
+msgid " Operation mode:\n"
+msgstr " Tryb pracy:\n"
+
+#: src/xz/message.c:1098
+msgid ""
+"  -z, --compress      force compression\n"
+"  -d, --decompress    force decompression\n"
+"  -t, --test          test compressed file integrity\n"
+"  -l, --list          list information about .xz files"
+msgstr ""
+"  -z, --compress      wymuszenie kompresji\n"
+"  -d, --decompress    wymuszenie dekompresji\n"
+"  -t, --test          sprawdzenie spójności plików skompresowanych\n"
+"  -l, --list          wypisanie informacji o plikach .xz"
+
+#: src/xz/message.c:1104
+msgid ""
+"\n"
+" Operation modifiers:\n"
+msgstr ""
+"\n"
+" Modyfikatory operacji:\n"
+
+#: src/xz/message.c:1107
+msgid ""
+"  -k, --keep          keep (don't delete) input files\n"
+"  -f, --force         force overwrite of output file and (de)compress links\n"
+"  -c, --stdout        write to standard output and don't delete input files"
+msgstr ""
+"  -k, --keep          zachowanie (nieusuwanie) plików wejściowych\n"
+"  -f, --force         nadpisywanie plików wyjściowych i (de)kompresja dowiązań\n"
+"  -c, --stdout        zapis na standardowe wyjście, nieusuwanie plików wej."
+
+#: src/xz/message.c:1113
+msgid ""
+"      --no-sparse     do not create sparse files when decompressing\n"
+"  -S, --suffix=.SUF   use the suffix `.SUF' on compressed files\n"
+"      --files[=FILE]  read filenames to process from FILE; if FILE is\n"
+"                      omitted, filenames are read from the standard input;\n"
+"                      filenames must be terminated with the newline character\n"
+"      --files0[=FILE] like --files but use the null character as terminator"
+msgstr ""
+"      --no-sparse     nietworzenie plików rzadkich podczas dekompresji\n"
+"  -S, --suffix=.ROZ   użycie rozszerzenia `.ROZ' dla plików skompresowanych\n"
+"      --files[=PLIK]  odczyt nazw plików do przetworzenia z PLIKU; jeśli PLIK\n"
+"                      nie został podany, nazwy są czytane ze standardowego\n"
+"                      wejścia; muszą być zakończone znakiem nowej linii\n"
+"      --files0[=PLIK] podobnie do --files, ale znakiem kończącym musi być NUL"
+
+#: src/xz/message.c:1121
+msgid ""
+"\n"
+" Basic file format and compression options:\n"
+msgstr ""
+"\n"
+" Podstawowe opcje formatu pliku i kompresji:\n"
+
+#: src/xz/message.c:1123
+msgid ""
+"  -F, --format=FMT    file format to encode or decode; possible values are\n"
+"                      `auto' (default), `xz', `lzma', and `raw'\n"
+"  -C, --check=CHECK   integrity check type: `none' (use with caution),\n"
+"                      `crc32', `crc64' (default), or `sha256'"
+msgstr ""
+"  -F, --format=FORM   format pliki do kodowania lub dekodowania; możliwe to\n"
+"                      `auto' (domyślny), `xz', 'lzma' i `raw'\n"
+"  -C, --check=TEST    typ kontroli spójności: `none' (ostrożnie!),\n"
+"                      `crc32', `crc64' (domyślny) lub `sha256'"
+
+#: src/xz/message.c:1130
+msgid ""
+"  -0 ... -9           compression preset; default is 6; take compressor *and*\n"
+"                      decompressor memory usage into account before using 7-9!"
+msgstr ""
+"  -0 ... -9           predefiniowane opcje kompresji; domyślna to 6; przed\n"
+"                      użyciem wartości 7-9 należy wziąć pod uwagę wykorzystanie\n"
+"                      pamięci przy kompresji *oraz* dekompresji!"
+
+#: src/xz/message.c:1134
+msgid ""
+"  -e, --extreme       try to improve compression ratio by using more CPU time;\n"
+"                      does not affect decompressor memory requirements"
+msgstr ""
+"  -e, --extreme       próba poprawy współczynnika kompresji z użyciem większej\n"
+"                      ilości czasu procesora; nie wpływa na wymagania\n"
+"                      pamięciowe dekompresora"
+
+#: src/xz/message.c:1139
+#, no-c-format
+msgid ""
+"      --memlimit-compress=LIMIT\n"
+"      --memlimit-decompress=LIMIT\n"
+"  -M, --memlimit=LIMIT\n"
+"                      set memory usage limit for compression, decompression,\n"
+"                      or both; LIMIT is in bytes, % of RAM, or 0 for defaults"
+msgstr ""
+"      --memlimit-compress=LIMIT\n"
+"      --memlimit-decompress=LIMIT\n"
+"  -M, --memlimit=LIMIT\n"
+"                      ustawienie limitu użycia pamięci dla kompresji,\n"
+"                      dekompresji lub obu; LIMIT jest w bajtach, % RAM lub 0\n"
+"                      dla limitów domyślnych"
+
+#: src/xz/message.c:1146
+msgid ""
+"      --no-adjust     if compression settings exceed the memory usage limit,\n"
+"                      give an error instead of adjusting the settings downwards"
+msgstr ""
+"      --no-adjust     jeśli ustawienia kompresji przekraczają limit użycia\n"
+"                      pamięci, zostanie zgłoszony błąd zamiast zmniejszania\n"
+"                      ustawień"
+
+#: src/xz/message.c:1152
+msgid ""
+"\n"
+" Custom filter chain for compression (alternative for using presets):"
+msgstr ""
+"\n"
+" Łańcuch własnych filtrów do kompresji (alternatywa do używania -0 .. -9):"
+
+#: src/xz/message.c:1161
+msgid ""
+"\n"
+"  --lzma1[=OPTS]      LZMA1 or LZMA2; OPTS is a comma-separated list of zero or\n"
+"  --lzma2[=OPTS]      more of the following options (valid values; default):\n"
+"                        preset=PRE reset options to a preset (0-9[e])\n"
+"                        dict=NUM   dictionary size (4KiB - 1536MiB; 8MiB)\n"
+"                        lc=NUM     number of literal context bits (0-4; 3)\n"
+"                        lp=NUM     number of literal position bits (0-4; 0)\n"
+"                        pb=NUM     number of position bits (0-4; 2)\n"
+"                        mode=MODE  compression mode (fast, normal; normal)\n"
+"                        nice=NUM   nice length of a match (2-273; 64)\n"
+"                        mf=NAME    match finder (hc3, hc4, bt2, bt3, bt4; bt4)\n"
+"                        depth=NUM  maximum search depth; 0=automatic (default)"
+msgstr ""
+"\n"
+"  --lzma1[=OPCJE]     LZMA1 lub LZMA2; OPCJE to oddzielona przecinkami lista\n"
+"  --lzma2[=OPCJE]     zera lub więcej następujących opcji (w nawiasach wartości\n"
+"                      poprawne; domyślne):\n"
+"                        preset=PRE ustawienie opcji na predefiniowane (0-9[e])\n"
+"                        dict=ILE   rozmiar słownika (4KiB - 1536MiB; 8MiB)\n"
+"                        lc=ILE     liczba bitów kontekstu literału (0-4; 3)\n"
+"                        lp=ILE     liczba bitów pozycji literału (0-4; 0)\n"
+"                        pp=ILE     liczba bitów pozycji (0-4; 2)\n"
+"                        mode=TRYB  tryb kompresji (fast, normal; normal)\n"
+"                        nice=ILE   długość dopasowania (2-273; 64)\n"
+"                        mf=NAZWA   dopasowywacz (hc3, hc4, bt2, bt3, bt4; bt4)\n"
+"                        depth=ILE  maks. głębokość szukania; 0=auto (domyślne)"
+
+#: src/xz/message.c:1176
+msgid ""
+"\n"
+"  --x86[=OPTS]        x86 BCJ filter (32-bit and 64-bit)\n"
+"  --powerpc[=OPTS]    PowerPC BCJ filter (big endian only)\n"
+"  --ia64[=OPTS]       IA-64 (Itanium) BCJ filter\n"
+"  --arm[=OPTS]        ARM BCJ filter (little endian only)\n"
+"  --armthumb[=OPTS]   ARM-Thumb BCJ filter (little endian only)\n"
+"  --sparc[=OPTS]      SPARC BCJ filter\n"
+"                      Valid OPTS for all BCJ filters:\n"
+"                        start=NUM  start offset for conversions (default=0)"
+msgstr ""
+"\n"
+"  --x86[=OPCJE]       Filtr BCJ x86 (32-bitowy lub 64-bitowy)\n"
+"  --powerpc[=OPCJE]   Filtr BCJ PowerPC (tylko big-endian)\n"
+"  --ia64[=OPCJE]      Filtr BCJ IA-64 (Itanium)\n"
+"  --arm[=OPCJE]       Filtr BCJ ARM (tylko little-endian)\n"
+"  --armthumb[=OPCJE]  Filtr BCJ ARM-Thumb (tylko little-endian)\n"
+"  --sparc[=OPCJE]     Filtr BCJ SPARC\n"
+"                      Poprawne OPCJE dla wszystkich filtrów BCJ:\n"
+"                        start=ILE  offset początku konwersji (domyślnie=0)"
+
+#: src/xz/message.c:1188
+msgid ""
+"\n"
+"  --delta[=OPTS]      Delta filter; valid OPTS (valid values; default):\n"
+"                        dist=NUM   distance between bytes being subtracted\n"
+"                                   from each other (1-256; 1)"
+msgstr ""
+"\n"
+"  --delta[=OPCJE]     Filtr delta; poprawne OPCJE (poprawne wart.; domyślne):\n"
+"                        dist=ILE   odległość między bajtami odejmowanymi od\n"
+"                                   siebie (1-256; 1)"
+
+#: src/xz/message.c:1196
+msgid ""
+"\n"
+" Other options:\n"
+msgstr ""
+"\n"
+" Inne opcje:\n"
+
+#: src/xz/message.c:1199
+msgid ""
+"  -q, --quiet         suppress warnings; specify twice to suppress errors too\n"
+"  -v, --verbose       be verbose; specify twice for even more verbose"
+msgstr ""
+"  -q, --quiet         pominięcie ostrzeżeń; dwukrotne podanie pomija też błędy\n"
+"  -v, --verbose       więcej informacji; dwukrotne podanie to jeszcze więcej"
+
+#: src/xz/message.c:1204
+msgid "  -Q, --no-warn       make warnings not affect the exit status"
+msgstr "  -Q, --no-warn       ostrzeżenia nie mają wpływu na status zakończenia"
+
+#: src/xz/message.c:1206
+msgid "      --robot         use machine-parsable messages (useful for scripts)"
+msgstr "      --robot         komunikaty w formacie dla maszyny (do skryptów)"
+
+#: src/xz/message.c:1209
+msgid ""
+"      --info-memory   display the total amount of RAM and the currently active\n"
+"                      memory usage limits, and exit"
+msgstr ""
+"      --info-memory   wyświetlenie całkowitej ilości pamięci RAM oraz aktualnie\n"
+"                      aktywnych limitów pamięci i zakończenie pracy"
+
+#: src/xz/message.c:1212
+msgid ""
+"  -h, --help          display the short help (lists only the basic options)\n"
+"  -H, --long-help     display this long help and exit"
+msgstr ""
+"  -h, --help          wyświetlenie krótkiego opisu (tylko podstawowe opcje)\n"
+"  -H, --long-help     wyświetlenie tego długiego opisu i zakończenie"
+
+#: src/xz/message.c:1216
+msgid ""
+"  -h, --help          display this short help and exit\n"
+"  -H, --long-help     display the long help (lists also the advanced options)"
+msgstr ""
+"  -h, --help          wyświetlenie tego krótkiego opisu i zakończenie\n"
+"  -H, --long-help     wyświetlenie długiego opisu (także opcje zaawansowane)"
+
+#: src/xz/message.c:1221
+msgid "  -V, --version       display the version number and exit"
+msgstr "  -V, --version       wyświetlenie informacji o wersji i zakończenie"
+
+#: src/xz/message.c:1223
+msgid ""
+"\n"
+"With no FILE, or when FILE is -, read standard input.\n"
+msgstr ""
+"\n"
+"Jeśli nie podano PLIKU lub PLIK to -, czytane jest standardowe wejście.\n"
+
+#. TRANSLATORS: This message indicates the bug reporting address
+#. for this package. Please add _another line_ saying
+#. "Report translation bugs to <...>\n" with the email or WWW
+#. address for translation bugs. Thanks.
+#: src/xz/message.c:1229
+#, c-format
+msgid "Report bugs to <%s> (in English or Finnish).\n"
+msgstr ""
+"Błędy prosimy zgłaszać na adres <%s>\n"
+"(w języku angielskim lub fińskim).\n"
+"Błędy w tłumaczeniu prosimy zgłaszać na adres\n"
+"<translation-team-pl@lists.sourceforge.net>.\n"
+
+#: src/xz/message.c:1231
+#, c-format
+msgid "%s home page: <%s>\n"
+msgstr "Strona domowa %s: <%s>\n"
+
+#: src/xz/options.c:86
+#, c-format
+msgid "%s: Options must be `name=value' pairs separated with commas"
+msgstr "%s: Opcje muszą być parami `nazwa=wartość' rozdzielonymi przecinkami"
+
+#: src/xz/options.c:93
+#, c-format
+msgid "%s: Invalid option name"
+msgstr "%s: Błędna nazwa opcji"
+
+#: src/xz/options.c:113
+#, c-format
+msgid "%s: Invalid option value"
+msgstr "%s: Błędna wartość opcji"
+
+#: src/xz/options.c:247
+#, c-format
+msgid "Unsupported LZMA1/LZMA2 preset: %s"
+msgstr "Nieobsługiwane ustawienie predefiniowane LZMA1/LZMA2: %s"
+
+#: src/xz/options.c:355
+msgid "The sum of lc and lp must not exceed 4"
+msgstr "Suma lc i lp nie może przekroczyć 4"
+
+#: src/xz/options.c:359
+#, c-format
+msgid "The selected match finder requires at least nice=%<PRIu32>"
+msgstr "Wybrany dopasowywacz wymaga przynajmniej nice=%<PRIu32>"
+
+#: src/xz/suffix.c:104 src/xz/suffix.c:189
+#, c-format
+msgid "%s: With --format=raw, --suffix=.SUF is required unless writing to stdout"
+msgstr "%s: Przy --format=raw i zapisie do pliku wymagana jest opcja --suffix=.ROZ"
+
+#: src/xz/suffix.c:124
+#, c-format
+msgid "%s: Filename has an unknown suffix, skipping"
+msgstr "%s: Nazwa pliku ma nieznane rozszerzenie, pominięto"
+
+#: src/xz/suffix.c:179
+#, c-format
+msgid "%s: File already has `%s' suffix, skipping"
+msgstr "%s: Plik już ma rozszerzenie `%s', pominięto"
+
+#: src/xz/suffix.c:230
+#, c-format
+msgid "%s: Invalid filename suffix"
+msgstr "%s: Błędne rozszerzenie nazwy pliku"
+
+#: src/xz/util.c:61
+#, c-format
+msgid "%s: Value is not a non-negative decimal integer"
+msgstr "%s: Wartość nie jest nieujemną liczbą całkowitą"
+
+#: src/xz/util.c:103
+#, c-format
+msgid "%s: Invalid multiplier suffix"
+msgstr "%s: Błędny przyrostek mnożnika"
+
+#: src/xz/util.c:105
+msgid "Valid suffixes are `KiB' (2^10), `MiB' (2^20), and `GiB' (2^30)."
+msgstr "Poprawne przyrostki to `KiB' (2^10), `MiB' (2^20) i `GiB' (2^30)."
+
+#: src/xz/util.c:122
+#, c-format
+msgid "Value of the option `%s' must be in the range [%<PRIu64>, %<PRIu64>]"
+msgstr "Wartość opcji `%s' musi być w przedziale [%<PRIu64>, %<PRIu64>]"
+
+#: src/xz/util.c:247
+msgid "Empty filename, skipping"
+msgstr "Pusta nazwa pliku, pominięto"
+
+#: src/xz/util.c:261
+msgid "Compressed data cannot be read from a terminal"
+msgstr "Dane skompresowane nie mogą być czytane z terminala"
+
+#: src/xz/util.c:274
+msgid "Compressed data cannot be written to a terminal"
+msgstr "Dane skompresowane nie mogą być zapisywane na terminal"
+
+#: src/common/tuklib_exit.c:39
+msgid "Writing to standard output failed"
+msgstr "Zapis na standardowe wyjście nie powiódł się"
+
+#: src/common/tuklib_exit.c:42
+msgid "Unknown error"
+msgstr "Nieznany błąd"

src/common/mythread.h

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       mythread.h
-/// \brief      Wrappers for threads
+/// \brief      Some threading related helper macros and functions
 //
 //  Author:     Lasse Collin
 //
@@ -14,29 +14,189 @@
 
 
 #ifdef HAVE_PTHREAD
-#	include <pthread.h>
 
-#	define mythread_once(func) \
-	do { \
-		static pthread_once_t once_ = PTHREAD_ONCE_INIT; \
-		pthread_once(&once_, &func); \
-	} while (0)
+////////////////////
+// Using pthreads //
+////////////////////
 
-#	define mythread_sigmask(how, set, oset) \
-		pthread_sigmask(how, set, oset)
+#include <pthread.h>
+#include <signal.h>
+#include <time.h>
+#include <unistd.h>
+
+
+/// \brief      Set the process signal mask
+///
+/// If threads are disabled, sigprocmask() is used instead
+/// of pthread_sigmask().
+#define mythread_sigmask(how, set, oset) \
+	pthread_sigmask(how, set, oset)
+
+
+/// \brief      Call the given function once
+///
+/// If threads are disabled, a thread-unsafe version is used.
+#define mythread_once(func) \
+do { \
+	static pthread_once_t once_ = PTHREAD_ONCE_INIT; \
+	pthread_once(&once_, &func); \
+} while (0)
+
+
+/// \brief      Lock a mutex for a duration of a block
+///
+/// Perform pthread_mutex_lock(&mutex) in the beginning of a block
+/// and pthread_mutex_unlock(&mutex) at the end of the block. "break"
+/// may be used to unlock the mutex and jump out of the block.
+/// mythread_sync blocks may be nested.
+///
+/// Example:
+///
+///     mythread_sync(mutex) {
+///         foo();
+///         if (some_error)
+///             break; // Skips bar()
+///         bar();
+///     }
+///
+/// At least GCC optimizes the loops completely away so it doesn't slow
+/// things down at all compared to plain pthread_mutex_lock(&mutex)
+/// and pthread_mutex_unlock(&mutex) calls.
+///
+#define mythread_sync(mutex) mythread_sync_helper(mutex, __LINE__)
+#define mythread_sync_helper(mutex, line) \
+	for (unsigned int mythread_i_ ## line = 0; \
+			mythread_i_ ## line \
+				? (pthread_mutex_unlock(&(mutex)), 0) \
+				: (pthread_mutex_lock(&(mutex)), 1); \
+			mythread_i_ ## line = 1) \
+		for (unsigned int mythread_j_ ## line = 0; \
+				!mythread_j_ ## line; \
+				mythread_j_ ## line = 1)
+
+
+typedef struct {
+	/// Condition variable
+	pthread_cond_t cond;
+
+	/// Clock ID (CLOCK_REALTIME or CLOCK_MONOTONIC) associated with
+	/// the condition variable
+	clockid_t clk_id;
+
+} mythread_cond;
+
+
+/// \brief      Initialize a condition variable to use CLOCK_MONOTONIC
+///
+/// Using CLOCK_MONOTONIC instead of the default CLOCK_REALTIME makes the
+/// timeout in pthread_cond_timedwait() work correctly also if system time
+/// is suddenly changed. Unfortunately CLOCK_MONOTONIC isn't available
+/// everywhere while the default CLOCK_REALTIME is, so the default is
+/// used if CLOCK_MONOTONIC isn't available.
+static inline int
+mythread_cond_init(mythread_cond *mycond)
+{
+#if defined(_POSIX_CLOCK_SELECTION) && defined(_POSIX_MONOTONIC_CLOCK)
+	struct timespec ts;
+	pthread_condattr_t condattr;
+
+	// POSIX doesn't seem to *require* that pthread_condattr_setclock()
+	// will fail if given an unsupported clock ID. Test that
+	// CLOCK_MONOTONIC really is supported using clock_gettime().
+	if (clock_gettime(CLOCK_MONOTONIC, &ts) == 0
+			&& pthread_condattr_init(&condattr) == 0) {
+		int ret = pthread_condattr_setclock(
+				&condattr, CLOCK_MONOTONIC);
+		if (ret == 0)
+			ret = pthread_cond_init(&mycond->cond, &condattr);
+
+		pthread_condattr_destroy(&condattr);
+
+		if (ret == 0) {
+			mycond->clk_id = CLOCK_MONOTONIC;
+			return 0;
+		}
+	}
+
+	// If anything above fails, fall back to the default CLOCK_REALTIME.
+#endif
+
+	mycond->clk_id = CLOCK_REALTIME;
+	return pthread_cond_init(&mycond->cond, NULL);
+}
+
+
+/// \brief      Convert relative time to absolute time for use with timed wait
+///
+/// The current time of the clock associated with the condition variable
+/// is added to the relative time in *ts.
+static inline void
+mythread_cond_abstime(const mythread_cond *mycond, struct timespec *ts)
+{
+	struct timespec now;
+	clock_gettime(mycond->clk_id, &now);
+
+	ts->tv_sec += now.tv_sec;
+	ts->tv_nsec += now.tv_nsec;
+
+	// tv_nsec must stay in the range [0, 999_999_999].
+	if (ts->tv_nsec >= 1000000000L) {
+		ts->tv_nsec -= 1000000000L;
+		++ts->tv_sec;
+	}
+
+	return;
+}
+
+
+#define mythread_cond_wait(mycondptr, mutexptr) \
+	pthread_cond_wait(&(mycondptr)->cond, mutexptr)
+
+#define mythread_cond_timedwait(mycondptr, mutexptr, abstimeptr) \
+	pthread_cond_timedwait(&(mycondptr)->cond, mutexptr, abstimeptr)
+
+#define mythread_cond_signal(mycondptr) \
+	pthread_cond_signal(&(mycondptr)->cond)
+
+#define mythread_cond_broadcast(mycondptr) \
+	pthread_cond_broadcast(&(mycondptr)->cond)
+
+#define mythread_cond_destroy(mycondptr) \
+	pthread_cond_destroy(&(mycondptr)->cond)
+
+
+/// \brief      Create a thread with all signals blocked
+static inline int
+mythread_create(pthread_t *thread, void *(*func)(void *arg), void *arg)
+{
+	sigset_t old;
+	sigset_t all;
+	sigfillset(&all);
+
+	pthread_sigmask(SIG_SETMASK, &all, &old);
+	const int ret = pthread_create(thread, NULL, func, arg);
+	pthread_sigmask(SIG_SETMASK, &old, NULL);
+
+	return ret;
+}
 
 #else
 
-#	define mythread_once(func) \
-	do { \
-		static bool once_ = false; \
-		if (!once_) { \
-			func(); \
-			once_ = true; \
-		} \
-	} while (0)
+//////////////////
+// No threading //
+//////////////////
 
-#	define mythread_sigmask(how, set, oset) \
-		sigprocmask(how, set, oset)
+#define mythread_sigmask(how, set, oset) \
+	sigprocmask(how, set, oset)
+
+
+#define mythread_once(func) \
+do { \
+	static bool once_ = false; \
+	if (!once_) { \
+		func(); \
+		once_ = true; \
+	} \
+} while (0)
 
 #endif

src/liblzma/Makefile.am

@@ -24,7 +24,7 @@ liblzma_la_CPPFLAGS = \
 	-I$(top_srcdir)/src/liblzma/simple \
 	-I$(top_srcdir)/src/common \
 	-DTUKLIB_SYMBOL_PREFIX=lzma_
-liblzma_la_LDFLAGS = -no-undefined -version-info 5:0:0
+liblzma_la_LDFLAGS = -no-undefined -version-info 5:99:0
 
 include $(srcdir)/common/Makefile.inc
 include $(srcdir)/check/Makefile.inc

src/liblzma/api/lzma/base.h

@@ -332,11 +332,19 @@ typedef enum {
  * malloc() and free(). C++ users should note that the custom memory
  * handling functions must not throw exceptions.
  *
- * liblzma doesn't make an internal copy of lzma_allocator. Thus, it is
- * OK to change these function pointers in the middle of the coding
- * process, but obviously it must be done carefully to make sure that the
- * replacement `free' can deallocate memory allocated by the earlier
- * `alloc' function(s).
+ * Single-threaded mode only: liblzma doesn't make an internal copy of
+ * lzma_allocator. Thus, it is OK to change these function pointers in
+ * the middle of the coding process, but obviously it must be done
+ * carefully to make sure that the replacement `free' can deallocate
+ * memory allocated by the earlier `alloc' function(s).
+ *
+ * Multithreaded mode: liblzma might internally store pointers to the
+ * lzma_allocator given via the lzma_stream structure. The application
+ * must not change the allocator pointer in lzma_stream or the contents
+ * of the pointed lzma_allocator structure until lzma_end() has been used
+ * to free the memory associated with that lzma_stream. The allocation
+ * functions might be called simultaneously from multiple threads, and
+ * thus they must be thread safe.
  */
 typedef struct {
 	/**

src/liblzma/api/lzma/block.h

@@ -483,6 +483,7 @@ extern LZMA_API(size_t) lzma_block_buffer_bound(size_t uncompressed_size)
  *
  * \return      - LZMA_OK: Encoding was successful.
  *              - LZMA_BUF_ERROR: Not enough output buffer space.
+ *              - LZMA_UNSUPPORTED_CHECK
  *              - LZMA_OPTIONS_ERROR
  *              - LZMA_MEM_ERROR
  *              - LZMA_DATA_ERROR

src/liblzma/api/lzma/container.h

@@ -60,12 +60,139 @@
 #define LZMA_PRESET_EXTREME       (UINT32_C(1) << 31)
 
 
+#ifdef LZMA_UNSTABLE /* Unstable API that may change. Use only for testing. */
+/**
+ * \brief       Multithreading options
+ */
+typedef struct {
+	/**
+	 * \brief       Flags
+	 *
+	 * Set this to zero if no flags are wanted.
+	 *
+	 * No flags are currently supported.
+	 */
+	uint32_t flags;
+
+	/**
+	 * \brief       Number of worker threads to use
+	 */
+	uint32_t threads;
+
+	/**
+	 * \brief       Maximum uncompressed size of a Block
+	 *
+	 * The encoder will start a new .xz Block every block_size bytes.
+	 * Using LZMA_FULL_FLUSH or LZMA_FULL_BARRIER with lzma_code()
+	 * the caller may tell liblzma to start a new Block earlier.
+	 *
+	 * With LZMA2, a recommended block size is 2-4 times the LZMA2
+	 * dictionary size. With very small dictionaries, it is recommended
+	 * to use at least 1 MiB block size for good compression ratio, even
+	 * if this is more than four times the dictionary size. Note that
+	 * these are only recommendations for typical use cases; feel free
+	 * to use other values. Just keep in mind that using a block size
+	 * less than the LZMA2 dictionary size is waste of RAM.
+	 *
+	 * Set this to 0 to let liblzma choose the block size depending
+	 * on the compression options. For LZMA2 it will be 3*dict_size
+	 * or 1 MiB, whichever is more.
+	 */
+	uint64_t block_size;
+
+	/**
+	 * \brief       Timeout to allow lzma_code() to return early
+	 *
+	 * Multithreading can make liblzma to consume input and produce
+	 * output in a very bursty way: it may first read a lot of input
+	 * to fill internal buffers, then no input or output occurs for
+	 * a while.
+	 *
+	 * In single-threaded mode, lzma_code() won't return until it has
+	 * either consumed all the input or filled the output buffer. If
+	 * this is done in multithreaded mode, it may cause a call
+	 * lzma_code() to take even tens of seconds, which isn't acceptable
+	 * in all applications.
+	 *
+	 * To avoid very long blocking times in lzma_code(), a timeout
+	 * (in milliseconds) may be set here. If lzma_code() would block
+	 * longer than this number of milliseconds, it will return with
+	 * LZMA_OK. Reasonable values are 100 ms or more. The xz command
+	 * line tool uses 300 ms.
+	 *
+	 * If long blocking times are fine for you, set timeout to a special
+	 * value of 0, which will disable the timeout mechanism and will make
+	 * lzma_code() block until all the input is consumed or the output
+	 * buffer has been filled.
+	 *
+	 * \note        Even with a timeout, lzma_code() might sometimes take
+	 *              somewhat long time to return. No timing guarantees
+	 *              are made.
+	 */
+	uint32_t timeout;
+
+	/**
+	 * \brief       Compression preset (level and possible flags)
+	 *
+	 * The preset is set just like with lzma_easy_encoder().
+	 * The preset is ignored if filters below is non-NULL.
+	 */
+	uint32_t preset;
+
+	/**
+	 * \brief       Filter chain (alternative to a preset)
+	 *
+	 * If this is NULL, the preset above is used. Otherwise the preset
+	 * is ignored and the filter chain specified here is used.
+	 */
+	const lzma_filter *filters;
+
+	/**
+	 * \brief       Integrity check type
+	 *
+	 * See check.h for available checks. The xz command line tool
+	 * defaults to LZMA_CHECK_CRC64, which is a good choice if you
+	 * are unsure.
+	 */
+	lzma_check check;
+
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. You should not touch these, because the names
+	 * of these variables may change. These are and will never be used
+	 * with the currently supported options, so it is safe to leave these
+	 * uninitialized.
+	 */
+	lzma_reserved_enum reserved_enum1;
+	lzma_reserved_enum reserved_enum2;
+	lzma_reserved_enum reserved_enum3;
+	uint32_t reserved_int1;
+	uint32_t reserved_int2;
+	uint32_t reserved_int3;
+	uint32_t reserved_int4;
+	uint64_t reserved_int5;
+	uint64_t reserved_int6;
+	uint64_t reserved_int7;
+	uint64_t reserved_int8;
+	void *reserved_ptr1;
+	void *reserved_ptr2;
+	void *reserved_ptr3;
+	void *reserved_ptr4;
+
+} lzma_mt;
+#endif
+
+
 /**
  * \brief       Calculate approximate memory usage of easy encoder
  *
  * This function is a wrapper for lzma_raw_encoder_memusage().
  *
  * \param       preset  Compression preset (level and possible flags)
+ *
+ * \return      Number of bytes of memory required for the given
+ *              preset when encoding. If an error occurs, for example
+ *              due to unsupported preset, UINT64_MAX is returned.
  */
 extern LZMA_API(uint64_t) lzma_easy_encoder_memusage(uint32_t preset)
 		lzma_nothrow lzma_attr_pure;
@@ -77,6 +204,11 @@ extern LZMA_API(uint64_t) lzma_easy_encoder_memusage(uint32_t preset)
  * This function is a wrapper for lzma_raw_decoder_memusage().
  *
  * \param       preset  Compression preset (level and possible flags)
+ *
+ * \return      Number of bytes of memory required to decompress a file
+ *              that was compressed using the given preset. If an error
+ *              occurs, for example due to unsupported preset, UINT64_MAX
+ *              is returned.
  */
 extern LZMA_API(uint64_t) lzma_easy_decoder_memusage(uint32_t preset)
 		lzma_nothrow lzma_attr_pure;
@@ -148,6 +280,7 @@ extern LZMA_API(lzma_ret) lzma_easy_encoder(
  *
  * \return      - LZMA_OK: Encoding was successful.
  *              - LZMA_BUF_ERROR: Not enough output buffer space.
+ *              - LZMA_UNSUPPORTED_CHECK
  *              - LZMA_OPTIONS_ERROR
  *              - LZMA_MEM_ERROR
  *              - LZMA_DATA_ERROR
@@ -171,6 +304,7 @@ extern LZMA_API(lzma_ret) lzma_easy_buffer_encode(
  *
  * \return      - LZMA_OK: Initialization was successful.
  *              - LZMA_MEM_ERROR
+ *              - LZMA_UNSUPPORTED_CHECK
  *              - LZMA_OPTIONS_ERROR
  *              - LZMA_PROG_ERROR
  */
@@ -179,6 +313,50 @@ extern LZMA_API(lzma_ret) lzma_stream_encoder(lzma_stream *strm,
 		lzma_nothrow lzma_attr_warn_unused_result;
 
 
+#ifdef LZMA_UNSTABLE /* Unstable API that may change. Use only for testing. */
+/**
+ * \brief       Calculate approximate memory usage of multithreaded .xz encoder
+ *
+ * Since doing the encoding in threaded mode doesn't affect the memory
+ * requirements of single-threaded decompressor, you can use
+ * lzma_easy_decoder_memusage(options->preset) or
+ * lzma_raw_decoder_memusage(options->filters) to calculate
+ * the decompressor memory requirements.
+ *
+ * \param       options Compression options
+ *
+ * \return      Number of bytes of memory required for encoding with the
+ *              given options. If an error occurs, for example due to
+ *              unsupported preset or filter chain, UINT64_MAX is returned.
+ */
+extern LZMA_API(uint64_t) lzma_stream_encoder_mt_memusage(
+		const lzma_mt *options) lzma_nothrow lzma_attr_pure;
+
+
+/**
+ * \brief       Initialize multithreaded .xz Stream encoder
+ *
+ * This provides the functionality of lzma_easy_encoder() and
+ * lzma_stream_encoder() as a single function for multithreaded use.
+ *
+ * TODO: For lzma_code(), only LZMA_RUN and LZMA_FINISH are currently
+ * supported. Support for other actions has been planned.
+ *
+ * \param       strm    Pointer to properly prepared lzma_stream
+ * \param       options Pointer to multithreaded compression options
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_UNSUPPORTED_CHECK
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern LZMA_API(lzma_ret) lzma_stream_encoder_mt(
+		lzma_stream *strm, const lzma_mt *options)
+		lzma_nothrow lzma_attr_warn_unused_result;
+#endif
+
+
 /**
  * \brief       Initialize .lzma encoder (legacy file format)
  *
@@ -250,6 +428,7 @@ extern LZMA_API(size_t) lzma_stream_buffer_bound(size_t uncompressed_size)
  *
  * \return      - LZMA_OK: Encoding was successful.
  *              - LZMA_BUF_ERROR: Not enough output buffer space.
+ *              - LZMA_UNSUPPORTED_CHECK
  *              - LZMA_OPTIONS_ERROR
  *              - LZMA_MEM_ERROR
  *              - LZMA_DATA_ERROR

src/liblzma/api/lzma/filter.h

@@ -131,7 +131,9 @@ extern LZMA_API(lzma_ret) lzma_filters_copy(const lzma_filter *src,
  *                          .id == LZMA_VLI_UNKNOWN.
  *
  * \return      Number of bytes of memory required for the given
- *              filter chain when encoding.
+ *              filter chain when encoding. If an error occurs,
+ *              for example due to unsupported filter chain,
+ *              UINT64_MAX is returned.
  */
 extern LZMA_API(uint64_t) lzma_raw_encoder_memusage(const lzma_filter *filters)
 		lzma_nothrow lzma_attr_pure;
@@ -148,7 +150,9 @@ extern LZMA_API(uint64_t) lzma_raw_encoder_memusage(const lzma_filter *filters)
  *                          .id == LZMA_VLI_UNKNOWN.
  *
  * \return      Number of bytes of memory required for the given
- *              filter chain when decoding.
+ *              filter chain when decoding. If an error occurs,
+ *              for example due to unsupported filter chain,
+ *              UINT64_MAX is returned.
  */
 extern LZMA_API(uint64_t) lzma_raw_decoder_memusage(const lzma_filter *filters)
 		lzma_nothrow lzma_attr_pure;

src/liblzma/api/lzma/version.h

@@ -21,9 +21,9 @@
  * Version number split into components
  */
 #define LZMA_VERSION_MAJOR 5
-#define LZMA_VERSION_MINOR 0
-#define LZMA_VERSION_PATCH 0
-#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE
+#define LZMA_VERSION_MINOR 1
+#define LZMA_VERSION_PATCH 1
+#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_ALPHA
 
 #ifndef LZMA_VERSION_COMMIT
 #	define LZMA_VERSION_COMMIT ""

src/liblzma/check/sha256.c

@@ -22,13 +22,13 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
+#include "check.h"
+
 // Avoid bogus warnings in transform().
-#if (__GNUC__ == 4 && __GNUC_MINOR__ >= 2) || __GNUC__ > 4
+#if TUKLIB_GNUC_REQ(4, 2)
 #	pragma GCC diagnostic ignored "-Wuninitialized"
 #endif
 
-#include "check.h"
-
 // At least on x86, GCC is able to optimize this to a rotate instruction.
 #define rotr_32(num, amount) ((num) >> (amount) | (num) << (32 - (amount)))
 

src/liblzma/common/Makefile.inc

@@ -38,9 +38,15 @@ liblzma_la_SOURCES += \
 	common/index_encoder.h \
 	common/stream_buffer_encoder.c \
 	common/stream_encoder.c \
-	common/stream_encoder.h \
 	common/stream_flags_encoder.c \
 	common/vli_encoder.c
+
+if COND_THREADS
+liblzma_la_SOURCES += \
+	common/outqueue.c \
+	common/outqueue.h \
+	common/stream_encoder_mt.c
+endif
 endif
 
 if COND_MAIN_DECODER

src/liblzma/common/alone_encoder.c

@@ -103,7 +103,7 @@ alone_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 	if (options->dict_size < LZMA_DICT_SIZE_MIN)
 		return LZMA_OPTIONS_ERROR;
 
-	// Round up to to the next 2^n or 2^n + 2^(n - 1) depending on which
+	// Round up to the next 2^n or 2^n + 2^(n - 1) depending on which
 	// one is the next unless it is UINT32_MAX. While the header would
 	// allow any 32-bit integer, we do this to keep the decoder of liblzma
 	// accepting the resulting files.

src/liblzma/common/block_buffer_encoder.c

@@ -226,16 +226,23 @@ lzma_block_buffer_encode(lzma_block *block, lzma_allocator *allocator,
 		const uint8_t *in, size_t in_size,
 		uint8_t *out, size_t *out_pos, size_t out_size)
 {
-	// Sanity checks
-	if (block == NULL || block->filters == NULL
-			|| (in == NULL && in_size != 0) || out == NULL
+	// Validate the arguments.
+	if (block == NULL || (in == NULL && in_size != 0) || out == NULL
 			|| out_pos == NULL || *out_pos > out_size)
 		return LZMA_PROG_ERROR;
 
-	// Check the version field.
+	// The contents of the structure may depend on the version so
+	// check the version before validating the contents of *block.
 	if (block->version != 0)
 		return LZMA_OPTIONS_ERROR;
 
+	if ((unsigned int)(block->check) > LZMA_CHECK_ID_MAX
+			|| block->filters == NULL)
+		return LZMA_PROG_ERROR;
+
+	if (!lzma_check_is_supported(block->check))
+		return LZMA_UNSUPPORTED_CHECK;
+
 	// Size of a Block has to be a multiple of four, so limit the size
 	// here already. This way we don't need to check it again when adding
 	// Block Padding.
@@ -243,8 +250,7 @@ lzma_block_buffer_encode(lzma_block *block, lzma_allocator *allocator,
 
 	// Get the size of the Check field.
 	const size_t check_size = lzma_check_size(block->check);
-	if (check_size == UINT32_MAX)
-		return LZMA_PROG_ERROR;
+	assert(check_size != UINT32_MAX);
 
 	// Reserve space for the Check field.
 	if (out_size - *out_pos <= check_size)

src/liblzma/common/block_encoder.c

@@ -161,6 +161,11 @@ lzma_block_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 {
 	lzma_next_coder_init(&lzma_block_encoder_init, next, allocator);
 
+	if (block == NULL)
+		return LZMA_PROG_ERROR;
+
+	// The contents of the structure may depend on the version so
+	// check the version first.
 	if (block->version != 0)
 		return LZMA_OPTIONS_ERROR;
 

src/liblzma/common/common.c

@@ -156,10 +156,8 @@ lzma_strm_init(lzma_stream *strm)
 		strm->internal->next = LZMA_NEXT_CODER_INIT;
 	}
 
-	strm->internal->supported_actions[LZMA_RUN] = false;
-	strm->internal->supported_actions[LZMA_SYNC_FLUSH] = false;
-	strm->internal->supported_actions[LZMA_FULL_FLUSH] = false;
-	strm->internal->supported_actions[LZMA_FINISH] = false;
+	memzero(strm->internal->supported_actions,
+			sizeof(strm->internal->supported_actions));
 	strm->internal->sequence = ISEQ_RUN;
 	strm->internal->allow_buf_error = false;
 
@@ -265,7 +263,9 @@ lzma_code(lzma_stream *strm, lzma_action action)
 
 	strm->internal->avail_in = strm->avail_in;
 
-	switch (ret) {
+	// Cast is needed to silence a warning about LZMA_TIMED_OUT, which
+	// isn't part of lzma_ret enumeration.
+	switch ((unsigned int)(ret)) {
 	case LZMA_OK:
 		// Don't return LZMA_BUF_ERROR when it happens the first time.
 		// This is to avoid returning LZMA_BUF_ERROR when avail_out
@@ -281,6 +281,11 @@ lzma_code(lzma_stream *strm, lzma_action action)
 		}
 		break;
 
+	case LZMA_TIMED_OUT:
+		strm->internal->allow_buf_error = false;
+		ret = LZMA_OK;
+		break;
+
 	case LZMA_STREAM_END:
 		if (strm->internal->sequence == ISEQ_SYNC_FLUSH
 				|| strm->internal->sequence == ISEQ_FULL_FLUSH)

src/liblzma/common/common.h

@@ -32,6 +32,8 @@
 
 #define LZMA_API(type) LZMA_API_EXPORT type LZMA_API_CALL
 
+#define LZMA_UNSTABLE
+
 #include "lzma.h"
 
 // These allow helping the compiler in some often-executed branches, whose
@@ -49,6 +51,13 @@
 #define LZMA_BUFFER_SIZE 4096
 
 
+/// Maximum number of worker threads within one multithreaded component.
+/// The limit exists solely to make it simpler to prevent integer overflows
+/// when allocating structures etc. This should be big enough for now...
+/// the code won't scale anywhere close to this number anyway.
+#define LZMA_THREADS_MAX 16384
+
+
 /// Starting value for memory usage estimates. Instead of calculating size
 /// of _every_ structure and taking into account malloc() overhead etc., we
 /// add a base size to all memory usage estimates. It's not very accurate
@@ -69,6 +78,13 @@
 	| LZMA_CONCATENATED )
 
 
+/// Special return value (lzma_ret) to indicate that a timeout was reached
+/// and lzma_code() must not return LZMA_BUF_ERROR. This is converted to
+/// LZMA_OK in lzma_code(). This is not in the lzma_ret enumeration because
+/// there's no need to have it in the public API.
+#define LZMA_TIMED_OUT 32
+
+
 /// Type of encoder/decoder specific data; the actual structure is defined
 /// differently in different coders.
 typedef struct lzma_coder_s lzma_coder;

src/liblzma/common/easy_encoder.c

@@ -11,7 +11,6 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #include "easy_preset.h"
-#include "stream_encoder.h"
 
 
 extern LZMA_API(lzma_ret)

src/liblzma/common/filter_common.c

@@ -43,7 +43,7 @@ static const struct {
 		.changes_size = true,
 	},
 #endif
-#ifdef HAVE_DECODER_LZMA2
+#if defined(HAVE_ENCODER_LZMA2) || defined(HAVE_DECODER_LZMA2)
 	{
 		.id = LZMA_FILTER_LZMA2,
 		.options_size = sizeof(lzma_options_lzma),
@@ -52,7 +52,7 @@ static const struct {
 		.changes_size = true,
 	},
 #endif
-#ifdef HAVE_DECODER_X86
+#if defined(HAVE_ENCODER_X86) || defined(HAVE_DECODER_X86)
 	{
 		.id = LZMA_FILTER_X86,
 		.options_size = sizeof(lzma_options_bcj),
@@ -70,7 +70,7 @@ static const struct {
 		.changes_size = false,
 	},
 #endif
-#ifdef HAVE_DECODER_IA64
+#if defined(HAVE_ENCODER_IA64) || defined(HAVE_DECODER_IA64)
 	{
 		.id = LZMA_FILTER_IA64,
 		.options_size = sizeof(lzma_options_bcj),

src/liblzma/common/filter_encoder.c

@@ -30,11 +30,11 @@ typedef struct {
 	/// invalid, UINT64_MAX is returned.
 	uint64_t (*memusage)(const void *options);
 
-	/// Calculates the minimum sane size for Blocks (or other types of
-	/// chunks) to which the input data can be split to make
-	/// multithreaded encoding possible. If this is NULL, it is assumed
-	/// that the encoder is fast enough with single thread.
-	lzma_vli (*chunk_size)(const void *options);
+	/// Calculates the recommended Uncompressed Size for .xz Blocks to
+	/// which the input data can be split to make multithreaded
+	/// encoding possible. If this is NULL, it is assumed that
+	/// the encoder is fast enough with single thread.
+	uint64_t (*block_size)(const void *options);
 
 	/// Tells the size of the Filter Properties field. If options are
 	/// invalid, UINT32_MAX is returned. If this is NULL, props_size_fixed
@@ -59,7 +59,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_LZMA1,
 		.init = &lzma_lzma_encoder_init,
 		.memusage = &lzma_lzma_encoder_memusage,
-		.chunk_size = NULL, // FIXME
+		.block_size = NULL, // FIXME
 		.props_size_get = NULL,
 		.props_size_fixed = 5,
 		.props_encode = &lzma_lzma_props_encode,
@@ -70,7 +70,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_LZMA2,
 		.init = &lzma_lzma2_encoder_init,
 		.memusage = &lzma_lzma2_encoder_memusage,
-		.chunk_size = NULL, // FIXME
+		.block_size = &lzma_lzma2_block_size, // FIXME
 		.props_size_get = NULL,
 		.props_size_fixed = 1,
 		.props_encode = &lzma_lzma2_props_encode,
@@ -81,7 +81,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_X86,
 		.init = &lzma_simple_x86_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -91,7 +91,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_POWERPC,
 		.init = &lzma_simple_powerpc_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -101,7 +101,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_IA64,
 		.init = &lzma_simple_ia64_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -111,7 +111,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_ARM,
 		.init = &lzma_simple_arm_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -121,7 +121,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_ARMTHUMB,
 		.init = &lzma_simple_armthumb_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -131,7 +131,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_SPARC,
 		.init = &lzma_simple_sparc_encoder_init,
 		.memusage = NULL,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = &lzma_simple_props_size,
 		.props_encode = &lzma_simple_props_encode,
 	},
@@ -141,7 +141,7 @@ static const lzma_filter_encoder encoders[] = {
 		.id = LZMA_FILTER_DELTA,
 		.init = &lzma_delta_encoder_init,
 		.memusage = &lzma_delta_coder_memusage,
-		.chunk_size = NULL,
+		.block_size = NULL,
 		.props_size_get = NULL,
 		.props_size_fixed = 1,
 		.props_encode = &lzma_delta_props_encode,
@@ -226,20 +226,19 @@ lzma_raw_encoder_memusage(const lzma_filter *filters)
 }
 
 
-/*
-extern LZMA_API(lzma_vli)
-lzma_chunk_size(const lzma_filter *filters)
+extern uint64_t
+lzma_mt_block_size(const lzma_filter *filters)
 {
-	lzma_vli max = 0;
+	uint64_t max = 0;
 
 	for (size_t i = 0; filters[i].id != LZMA_VLI_UNKNOWN; ++i) {
 		const lzma_filter_encoder *const fe
 				= encoder_find(filters[i].id);
-		if (fe->chunk_size != NULL) {
-			const lzma_vli size
-					= fe->chunk_size(filters[i].options);
-			if (size == LZMA_VLI_UNKNOWN)
-				return LZMA_VLI_UNKNOWN;
+		if (fe->block_size != NULL) {
+			const uint64_t size
+					= fe->block_size(filters[i].options);
+			if (size == 0)
+				return 0;
 
 			if (size > max)
 				max = size;
@@ -248,7 +247,6 @@ lzma_chunk_size(const lzma_filter *filters)
 
 	return max;
 }
-*/
 
 
 extern LZMA_API(lzma_ret)

src/liblzma/common/filter_encoder.h

@@ -16,8 +16,8 @@
 #include "common.h"
 
 
-// FIXME: Might become a part of the public API once finished.
-// extern lzma_vli lzma_chunk_size(const lzma_filter *filters);
+// FIXME: Might become a part of the public API.
+extern uint64_t lzma_mt_block_size(const lzma_filter *filters);
 
 
 extern lzma_ret lzma_raw_encoder_init(

src/liblzma/common/outqueue.c

@@ -0,0 +1,180 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       outqueue.c
+/// \brief      Output queue handling in multithreaded coding
+//
+//  Author:     Lasse Collin
+//
+//  This file has been put into the public domain.
+//  You can do whatever you want with this file.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "outqueue.h"
+
+
+/// This is to ease integer overflow checking: We may allocate up to
+/// 2 * LZMA_THREADS_MAX buffers and we need some extra memory for other
+/// data structures (that's the second /2).
+#define BUF_SIZE_MAX (UINT64_MAX / LZMA_THREADS_MAX / 2 / 2)
+
+
+static lzma_ret
+get_options(uint64_t *bufs_alloc_size, uint32_t *bufs_count,
+		uint64_t buf_size_max, uint32_t threads)
+{
+	if (threads > LZMA_THREADS_MAX || buf_size_max > BUF_SIZE_MAX)
+		return LZMA_OPTIONS_ERROR;
+
+	// The number of buffers is twice the number of threads.
+	// This wastes RAM but keeps the threads busy when buffers
+	// finish out of order.
+	//
+	// NOTE: If this is changed, update BUF_SIZE_MAX too.
+	*bufs_count = threads * 2;
+	*bufs_alloc_size = *bufs_count * buf_size_max;
+
+	return LZMA_OK;
+}
+
+
+extern uint64_t
+lzma_outq_memusage(uint64_t buf_size_max, uint32_t threads)
+{
+	uint64_t bufs_alloc_size;
+	uint32_t bufs_count;
+
+	if (get_options(&bufs_alloc_size, &bufs_count, buf_size_max, threads)
+			!= LZMA_OK)
+		return UINT64_MAX;
+
+	return sizeof(lzma_outq) + bufs_count * sizeof(lzma_outbuf)
+			+ bufs_alloc_size;
+}
+
+
+extern lzma_ret
+lzma_outq_init(lzma_outq *outq, lzma_allocator *allocator,
+		uint64_t buf_size_max, uint32_t threads)
+{
+	uint64_t bufs_alloc_size;
+	uint32_t bufs_count;
+
+	// Set bufs_count and bufs_alloc_size.
+	return_if_error(get_options(&bufs_alloc_size, &bufs_count,
+			buf_size_max, threads));
+
+	// Allocate memory if needed.
+	if (outq->buf_size_max != buf_size_max
+			|| outq->bufs_allocated != bufs_count) {
+		lzma_outq_end(outq, allocator);
+
+#if SIZE_MAX < UINT64_MAX
+		if (bufs_alloc_size > SIZE_MAX)
+			return LZMA_MEM_ERROR;
+#endif
+
+		outq->bufs = lzma_alloc(bufs_count * sizeof(lzma_outbuf),
+				allocator);
+		outq->bufs_mem = lzma_alloc((size_t)(bufs_alloc_size),
+				allocator);
+
+		if (outq->bufs == NULL || outq->bufs_mem == NULL) {
+			lzma_outq_end(outq, allocator);
+			return LZMA_MEM_ERROR;
+		}
+	}
+
+	// Initialize the rest of the main structure. Initialization of
+	// outq->bufs[] is done when they are actually needed.
+	outq->buf_size_max = (size_t)(buf_size_max);
+	outq->bufs_allocated = bufs_count;
+	outq->bufs_pos = 0;
+	outq->bufs_used = 0;
+	outq->read_pos = 0;
+
+	return LZMA_OK;
+}
+
+
+extern void
+lzma_outq_end(lzma_outq *outq, lzma_allocator *allocator)
+{
+	lzma_free(outq->bufs, allocator);
+	lzma_free(outq->bufs_mem, allocator);
+	return;
+}
+
+
+extern lzma_outbuf *
+lzma_outq_get_buf(lzma_outq *outq)
+{
+	// Caller must have checked it with lzma_outq_has_buf().
+	assert(outq->bufs_used < outq->bufs_allocated);
+
+	// Initialize the new buffer.
+	lzma_outbuf *buf = &outq->bufs[outq->bufs_pos];
+	buf->buf = outq->bufs_mem + outq->bufs_pos * outq->buf_size_max;
+	buf->size = 0;
+	buf->finished = false;
+
+	// Update the queue state.
+	if (++outq->bufs_pos == outq->bufs_allocated)
+		outq->bufs_pos = 0;
+
+	++outq->bufs_used;
+
+	return buf;
+}
+
+
+extern bool
+lzma_outq_is_readable(const lzma_outq *outq)
+{
+	uint32_t i = outq->bufs_pos - outq->bufs_used;
+	if (outq->bufs_pos < outq->bufs_used)
+		i += outq->bufs_allocated;
+
+	return outq->bufs[i].finished;
+}
+
+
+extern lzma_ret
+lzma_outq_read(lzma_outq *restrict outq, uint8_t *restrict out,
+		size_t *restrict out_pos, size_t out_size,
+		lzma_vli *restrict unpadded_size,
+		lzma_vli *restrict uncompressed_size)
+{
+	// There must be at least one buffer from which to read.
+	if (outq->bufs_used == 0)
+		return LZMA_OK;
+
+	// Get the buffer.
+	uint32_t i = outq->bufs_pos - outq->bufs_used;
+	if (outq->bufs_pos < outq->bufs_used)
+		i += outq->bufs_allocated;
+
+	lzma_outbuf *buf = &outq->bufs[i];
+
+	// If it isn't finished yet, we cannot read from it.
+	if (!buf->finished)
+		return LZMA_OK;
+
+	// Copy from the buffer to output.
+	lzma_bufcpy(buf->buf, &outq->read_pos, buf->size,
+			out, out_pos, out_size);
+
+	// Return if we didn't get all the data from the buffer.
+	if (outq->read_pos < buf->size)
+		return LZMA_OK;
+
+	// The buffer was finished. Tell the caller its size information.
+	*unpadded_size = buf->unpadded_size;
+	*uncompressed_size = buf->uncompressed_size;
+
+	// Free this buffer for further use.
+	--outq->bufs_used;
+	outq->read_pos = 0;
+
+	return LZMA_STREAM_END;
+}

src/liblzma/common/outqueue.h

@@ -0,0 +1,155 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       outqueue.h
+/// \brief      Output queue handling in multithreaded coding
+//
+//  Author:     Lasse Collin
+//
+//  This file has been put into the public domain.
+//  You can do whatever you want with this file.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "common.h"
+
+
+/// Output buffer for a single thread
+typedef struct {
+	/// Pointer to the output buffer of lzma_outq.buf_size_max bytes
+	uint8_t *buf;
+
+	/// Amount of data written to buf
+	size_t size;
+
+	/// Additional size information
+	lzma_vli unpadded_size;
+	lzma_vli uncompressed_size;
+
+	/// True when no more data will be written into this buffer.
+	///
+	/// \note       This is read by another thread and thus access
+	///             to this variable needs a mutex.
+	bool finished;
+
+} lzma_outbuf;
+
+
+typedef struct {
+	/// Array of buffers that are used cyclically.
+	lzma_outbuf *bufs;
+
+	/// Memory allocated for all the buffers
+	uint8_t *bufs_mem;
+
+	/// Amount of buffer space available in each buffer
+	size_t buf_size_max;
+
+	/// Number of buffers allocated
+	uint32_t bufs_allocated;
+
+	/// Position in the bufs array. The next buffer to be taken
+	/// into use is bufs[bufs_pos].
+	uint32_t bufs_pos;
+
+	/// Number of buffers in use
+	uint32_t bufs_used;
+
+	/// Position in the buffer in lzma_outq_read()
+	size_t read_pos;
+
+} lzma_outq;
+
+
+/**
+ * \brief       Calculate the memory usage of an output queue
+ *
+ * \return      Approximate memory usage in bytes or UINT64_MAX on error.
+ */
+extern uint64_t lzma_outq_memusage(uint64_t buf_size_max, uint32_t threads);
+
+
+/// \brief      Initialize an output queue
+///
+/// \param      outq            Pointer to an output queue. Before calling
+///                             this function the first time, *outq should
+///                             have been zeroed with memzero() so that this
+///                             function knows that there are no previous
+///                             allocations to free.
+/// \param      allocator       Pointer to allocator or NULL
+/// \param      buf_size_max    Maximum amount of data that a single buffer
+///                             in the queue may need to store.
+/// \param      threads         Number of buffers that may be in use
+///                             concurrently. Note that more than this number
+///                             of buffers will actually get allocated to
+///                             improve performance when buffers finish
+///                             out of order.
+///
+/// \return     - LZMA_OK
+///             - LZMA_MEM_ERROR
+///
+extern lzma_ret lzma_outq_init(lzma_outq *outq, lzma_allocator *allocator,
+		uint64_t buf_size_max, uint32_t threads);
+
+
+/// \brief      Free the memory associated with the output queue
+extern void lzma_outq_end(lzma_outq *outq, lzma_allocator *allocator);
+
+
+/// \brief      Get a new buffer
+///
+/// lzma_outq_has_buf() must be used to check that there is a buffer
+/// available before calling lzma_outq_get_buf().
+///
+extern lzma_outbuf *lzma_outq_get_buf(lzma_outq *outq);
+
+
+/// \brief      Test if there is data ready to be read
+///
+/// Call to this function must be protected with the same mutex that
+/// is used to protect lzma_outbuf.finished.
+///
+extern bool lzma_outq_is_readable(const lzma_outq *outq);
+
+
+/// \brief      Read finished data
+///
+/// \param      outq            Pointer to an output queue
+/// \param      out             Beginning of the output buffer
+/// \param      out_pos         The next byte will be written to
+///                             out[*out_pos].
+/// \param      out_size        Size of the out buffer; the first byte into
+///                             which no data is written to is out[out_size].
+/// \param      unpadded_size   Unpadded Size from the Block encoder
+/// \param      uncompressed_size Uncompressed Size from the Block encoder
+///
+/// \return     - LZMA: All OK. Either no data was available or the buffer
+///               being read didn't become empty yet.
+///             - LZMA_STREAM_END: The buffer being read was finished.
+///               *unpadded_size and *uncompressed_size were set.
+///
+/// \note       This reads lzma_outbuf.finished variables and thus call
+///             to this function needs to be protected with a mutex.
+///
+extern lzma_ret lzma_outq_read(lzma_outq *restrict outq,
+		uint8_t *restrict out, size_t *restrict out_pos,
+		size_t out_size, lzma_vli *restrict unpadded_size,
+		lzma_vli *restrict uncompressed_size);
+
+
+/// \brief      Test if there is at least one buffer free
+///
+/// This must be used before getting a new buffer with lzma_outq_get_buf().
+///
+static inline bool
+lzma_outq_has_buf(const lzma_outq *outq)
+{
+	return outq->bufs_used < outq->bufs_allocated;
+}
+
+
+/// \brief      Test if the queue is completely empty
+static inline bool
+lzma_outq_is_empty(const lzma_outq *outq)
+{
+	return outq->bufs_used == 0;
+}

src/liblzma/common/stream_buffer_encoder.c

@@ -51,6 +51,9 @@ lzma_stream_buffer_encode(lzma_filter *filters, lzma_check check,
 			|| out_pos_ptr == NULL || *out_pos_ptr > out_size)
 		return LZMA_PROG_ERROR;
 
+	if (!lzma_check_is_supported(check))
+		return LZMA_UNSUPPORTED_CHECK;
+
 	// Note for the paranoids: Index encoder prevents the Stream from
 	// getting too big and still being accepted with LZMA_OK, and Block
 	// encoder catches if the input is too big. So we don't need to
@@ -81,26 +84,32 @@ lzma_stream_buffer_encode(lzma_filter *filters, lzma_check check,
 
 	out_pos += LZMA_STREAM_HEADER_SIZE;
 
-	// Block
+	// Encode a Block but only if there is at least one byte of input.
 	lzma_block block = {
 		.version = 0,
 		.check = check,
 		.filters = filters,
 	};
 
-	return_if_error(lzma_block_buffer_encode(&block, allocator,
-			in, in_size, out, &out_pos, out_size));
+	if (in_size > 0)
+		return_if_error(lzma_block_buffer_encode(&block, allocator,
+				in, in_size, out, &out_pos, out_size));
 
 	// Index
 	{
-		// Create an Index with one Record.
+		// Create an Index. It will have one Record if there was
+		// at least one byte of input to encode. Otherwise the
+		// Index will be empty.
 		lzma_index *i = lzma_index_init(allocator);
 		if (i == NULL)
 			return LZMA_MEM_ERROR;
 
-		lzma_ret ret = lzma_index_append(i, allocator,
-				lzma_block_unpadded_size(&block),
-				block.uncompressed_size);
+		lzma_ret ret = LZMA_OK;
+
+		if (in_size > 0)
+			ret = lzma_index_append(i, allocator,
+					lzma_block_unpadded_size(&block),
+					block.uncompressed_size);
 
 		// If adding the Record was successful, encode the Index
 		// and get its size which will be stored into Stream Footer.

src/liblzma/common/stream_encoder.c

@@ -10,7 +10,6 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#include "stream_encoder.h"
 #include "block_encoder.h"
 #include "index_encoder.h"
 
@@ -26,7 +25,7 @@ struct lzma_coder_s {
 	} sequence;
 
 	/// True if Block encoder has been initialized by
-	/// lzma_stream_encoder_init() or stream_encoder_update()
+	/// stream_encoder_init() or stream_encoder_update()
 	/// and thus doesn't need to be initialized in stream_encode().
 	bool block_encoder_is_initialized;
 
@@ -126,7 +125,7 @@ stream_encode(lzma_coder *coder, lzma_allocator *allocator,
 		}
 
 		// Initialize the Block encoder unless it was already
-		// initialized by lzma_stream_encoder_init() or
+		// initialized by stream_encoder_init() or
 		// stream_encoder_update().
 		if (!coder->block_encoder_is_initialized)
 			return_if_error(block_encoder_init(coder, allocator));
@@ -262,11 +261,11 @@ stream_encoder_update(lzma_coder *coder, lzma_allocator *allocator,
 }
 
 
-extern lzma_ret
-lzma_stream_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
+static lzma_ret
+stream_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 		const lzma_filter *filters, lzma_check check)
 {
-	lzma_next_coder_init(&lzma_stream_encoder_init, next, allocator);
+	lzma_next_coder_init(&stream_encoder_init, next, allocator);
 
 	if (filters == NULL)
 		return LZMA_PROG_ERROR;
@@ -280,6 +279,7 @@ lzma_stream_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 		next->end = &stream_encoder_end;
 		next->update = &stream_encoder_update;
 
+		next->coder->filters[0].id = LZMA_VLI_UNKNOWN;
 		next->coder->block_encoder = LZMA_NEXT_CODER_INIT;
 		next->coder->index_encoder = LZMA_NEXT_CODER_INIT;
 		next->coder->index = NULL;
@@ -289,7 +289,6 @@ lzma_stream_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 	next->coder->sequence = SEQ_STREAM_HEADER;
 	next->coder->block_options.version = 0;
 	next->coder->block_options.check = check;
-	next->coder->filters[0].id = LZMA_VLI_UNKNOWN;
 
 	// Initialize the Index
 	lzma_index_end(next->coder->index, allocator);
@@ -320,7 +319,7 @@ extern LZMA_API(lzma_ret)
 lzma_stream_encoder(lzma_stream *strm,
 		const lzma_filter *filters, lzma_check check)
 {
-	lzma_next_strm_init(lzma_stream_encoder_init, strm, filters, check);
+	lzma_next_strm_init(stream_encoder_init, strm, filters, check);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
 	strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;

src/liblzma/common/stream_encoder.h

@@ -1,23 +0,0 @@
-///////////////////////////////////////////////////////////////////////////////
-//
-/// \file       stream_encoder.h
-/// \brief      Encodes .xz Streams
-//
-//  Author:     Lasse Collin
-//
-//  This file has been put into the public domain.
-//  You can do whatever you want with this file.
-//
-///////////////////////////////////////////////////////////////////////////////
-
-#ifndef LZMA_STREAM_ENCODER_H
-#define LZMA_STREAM_ENCODER_H
-
-#include "common.h"
-
-
-extern lzma_ret lzma_stream_encoder_init(
-		lzma_next_coder *next, lzma_allocator *allocator,
-		const lzma_filter *filters, lzma_check check);
-
-#endif

src/liblzma/lzma/fastpos.h

@@ -14,15 +14,15 @@
 #ifndef LZMA_FASTPOS_H
 #define LZMA_FASTPOS_H
 
-// LZMA encodes match distances (positions) by storing the highest two
-// bits using a six-bit value [0, 63], and then the missing lower bits.
-// Dictionary size is also stored using this encoding in the new .lzma
+// LZMA encodes match distances by storing the highest two bits using
+// a six-bit value [0, 63], and then the missing lower bits.
+// Dictionary size is also stored using this encoding in the .xz
 // file format header.
 //
 // fastpos.h provides a way to quickly find out the correct six-bit
 // values. The following table gives some examples of this encoding:
 //
-//      pos   return
+//     dist   return
 //       0       0
 //       1       1
 //       2       2
@@ -48,10 +48,10 @@
 // Provided functions or macros
 // ----------------------------
 //
-// get_pos_slot(pos) is the basic version. get_pos_slot_2(pos)
-// assumes that pos >= FULL_DISTANCES, thus the result is at least
-// FULL_DISTANCES_BITS * 2. Using get_pos_slot(pos) instead of
-// get_pos_slot_2(pos) would give the same result, but get_pos_slot_2(pos)
+// get_dist_slot(dist) is the basic version. get_dist_slot_2(dist)
+// assumes that dist >= FULL_DISTANCES, thus the result is at least
+// FULL_DISTANCES_BITS * 2. Using get_dist_slot(dist) instead of
+// get_dist_slot_2(dist) would give the same result, but get_dist_slot_2(dist)
 // should be tiny bit faster due to the assumption being made.
 //
 //
@@ -76,13 +76,14 @@
 // slightly faster, but sometimes it is a lot slower.
 
 #ifdef HAVE_SMALL
-#	define get_pos_slot(pos) ((pos) <= 4 ? (pos) : get_pos_slot_2(pos))
+#	define get_dist_slot(dist) \
+		((dist) <= 4 ? (dist) : get_dist_slot_2(dist))
 
 static inline uint32_t
-get_pos_slot_2(uint32_t pos)
+get_dist_slot_2(uint32_t dist)
 {
-	const uint32_t i = bsr32(pos);
-	return (i + i) + ((pos >> (i - 1)) & 1);
+	const uint32_t i = bsr32(dist);
+	return (i + i) + ((dist >> (i - 1)) & 1);
 }
 
 
@@ -99,39 +100,39 @@ extern const uint8_t lzma_fastpos[1 << FASTPOS_BITS];
 #define fastpos_limit(extra, n) \
 	(UINT32_C(1) << (FASTPOS_BITS + fastpos_shift(extra, n)))
 
-#define fastpos_result(pos, extra, n) \
-	lzma_fastpos[(pos) >> fastpos_shift(extra, n)] \
+#define fastpos_result(dist, extra, n) \
+	lzma_fastpos[(dist) >> fastpos_shift(extra, n)] \
 			+ 2 * fastpos_shift(extra, n)
 
 
 static inline uint32_t
-get_pos_slot(uint32_t pos)
+get_dist_slot(uint32_t dist)
 {
 	// If it is small enough, we can pick the result directly from
 	// the precalculated table.
-	if (pos < fastpos_limit(0, 0))
-		return lzma_fastpos[pos];
+	if (dist < fastpos_limit(0, 0))
+		return lzma_fastpos[dist];
 
-	if (pos < fastpos_limit(0, 1))
-		return fastpos_result(pos, 0, 1);
+	if (dist < fastpos_limit(0, 1))
+		return fastpos_result(dist, 0, 1);
 
-	return fastpos_result(pos, 0, 2);
+	return fastpos_result(dist, 0, 2);
 }
 
 
 #ifdef FULL_DISTANCES_BITS
 static inline uint32_t
-get_pos_slot_2(uint32_t pos)
+get_dist_slot_2(uint32_t dist)
 {
-	assert(pos >= FULL_DISTANCES);
+	assert(dist >= FULL_DISTANCES);
 
-	if (pos < fastpos_limit(FULL_DISTANCES_BITS - 1, 0))
-		return fastpos_result(pos, FULL_DISTANCES_BITS - 1, 0);
+	if (dist < fastpos_limit(FULL_DISTANCES_BITS - 1, 0))
+		return fastpos_result(dist, FULL_DISTANCES_BITS - 1, 0);
 
-	if (pos < fastpos_limit(FULL_DISTANCES_BITS - 1, 1))
-		return fastpos_result(pos, FULL_DISTANCES_BITS - 1, 1);
+	if (dist < fastpos_limit(FULL_DISTANCES_BITS - 1, 1))
+		return fastpos_result(dist, FULL_DISTANCES_BITS - 1, 1);
 
-	return fastpos_result(pos, FULL_DISTANCES_BITS - 1, 2);
+	return fastpos_result(dist, FULL_DISTANCES_BITS - 1, 2);
 }
 #endif
 

src/liblzma/lzma/lzma2_decoder.c

@@ -67,6 +67,10 @@ lzma2_decode(lzma_coder *restrict coder, lzma_dict *restrict dict,
 		const uint32_t control = in[*in_pos];
 		++*in_pos;
 
+		// End marker
+		if (control == 0x00)
+			return LZMA_STREAM_END;
+
 		if (control >= 0xE0 || control == 1) {
 			// Dictionary reset implies that next LZMA chunk has
 			// to set new properties.
@@ -104,10 +108,6 @@ lzma2_decode(lzma_coder *restrict coder, lzma_dict *restrict dict,
 							&coder->options);
 			}
 		} else {
-			// End marker
-			if (control == 0x00)
-				return LZMA_STREAM_END;
-
 			// Invalid control values
 			if (control > 2)
 				return LZMA_DATA_ERROR;

src/liblzma/lzma/lzma2_encoder.c

@@ -374,7 +374,7 @@ lzma_lzma2_props_encode(const void *options, uint8_t *out)
 	const lzma_options_lzma *const opt = options;
 	uint32_t d = my_max(opt->dict_size, LZMA_DICT_SIZE_MIN);
 
-	// Round up to to the next 2^n - 1 or 2^n + 2^(n - 1) - 1 depending
+	// Round up to the next 2^n - 1 or 2^n + 2^(n - 1) - 1 depending
 	// on which one is the next:
 	--d;
 	d |= d >> 2;
@@ -387,7 +387,17 @@ lzma_lzma2_props_encode(const void *options, uint8_t *out)
 	if (d == UINT32_MAX)
 		out[0] = 40;
 	else
-		out[0] = get_pos_slot(d + 1) - 24;
+		out[0] = get_dist_slot(d + 1) - 24;
 
 	return LZMA_OK;
 }
+
+
+extern uint64_t
+lzma_lzma2_block_size(const void *options)
+{
+	const lzma_options_lzma *const opt = options;
+
+	// Use at least 1 MiB to keep compression ratio better.
+	return my_max((uint64_t)(opt->dict_size) * 3, UINT64_C(1) << 20);
+}

src/liblzma/lzma/lzma2_encoder.h

@@ -38,4 +38,6 @@ extern uint64_t lzma_lzma2_encoder_memusage(const void *options);
 
 extern lzma_ret lzma_lzma2_props_encode(const void *options, uint8_t *out);
 
+extern uint64_t lzma_lzma2_block_size(const void *options);
+
 #endif

src/liblzma/lzma/lzma_common.h

@@ -171,53 +171,54 @@ literal_init(probability (*probs)[LITERAL_CODER_SIZE],
 // Match distance //
 ////////////////////
 
-// Different set of probabilities is used for match distances that have very
+// Different sets of probabilities are used for match distances that have very
 // short match length: Lengths of 2, 3, and 4 bytes have a separate set of
 // probabilities for each length. The matches with longer length use a shared
 // set of probabilities.
-#define LEN_TO_POS_STATES 4
+#define DIST_STATES 4
 
 // Macro to get the index of the appropriate probability array.
-#define get_len_to_pos_state(len) \
-	((len) < LEN_TO_POS_STATES + MATCH_LEN_MIN \
+#define get_dist_state(len) \
+	((len) < DIST_STATES + MATCH_LEN_MIN \
 		? (len) - MATCH_LEN_MIN \
-		: LEN_TO_POS_STATES - 1)
+		: DIST_STATES - 1)
 
-// The highest two bits of a match distance (pos slot) are encoded using six
-// bits. See fastpos.h for more explanation.
-#define POS_SLOT_BITS 6
-#define POS_SLOTS (1 << POS_SLOT_BITS)
+// The highest two bits of a match distance (distance slot) are encoded
+// using six bits. See fastpos.h for more explanation.
+#define DIST_SLOT_BITS 6
+#define DIST_SLOTS (1 << DIST_SLOT_BITS)
 
 // Match distances up to 127 are fully encoded using probabilities. Since
-// the highest two bits (pos slot) are always encoded using six bits, the
-// distances 0-3 don't need any additional bits to encode, since the pos
-// slot itself is the same as the actual distance. START_POS_MODEL_INDEX
-// indicates the first pos slot where at least one additional bit is needed.
-#define START_POS_MODEL_INDEX 4
+// the highest two bits (distance slot) are always encoded using six bits,
+// the distances 0-3 don't need any additional bits to encode, since the
+// distance slot itself is the same as the actual distance. DIST_MODEL_START
+// indicates the first distance slot where at least one additional bit is
+// needed.
+#define DIST_MODEL_START 4
 
 // Match distances greater than 127 are encoded in three pieces:
-//   - pos slot: the highest two bits
+//   - distance slot: the highest two bits
 //   - direct bits: 2-26 bits below the highest two bits
 //   - alignment bits: four lowest bits
 //
 // Direct bits don't use any probabilities.
 //
-// The pos slot value of 14 is for distances 128-191 (see the table in
+// The distance slot value of 14 is for distances 128-191 (see the table in
 // fastpos.h to understand why).
-#define END_POS_MODEL_INDEX 14
+#define DIST_MODEL_END 14
 
-// Pos slots that indicate a distance <= 127.
-#define FULL_DISTANCES_BITS (END_POS_MODEL_INDEX / 2)
+// Distance slots that indicate a distance <= 127.
+#define FULL_DISTANCES_BITS (DIST_MODEL_END / 2)
 #define FULL_DISTANCES (1 << FULL_DISTANCES_BITS)
 
 // For match distances greater than 127, only the highest two bits and the
 // lowest four bits (alignment) is encoded using probabilities.
 #define ALIGN_BITS 4
-#define ALIGN_TABLE_SIZE (1 << ALIGN_BITS)
-#define ALIGN_MASK (ALIGN_TABLE_SIZE - 1)
+#define ALIGN_SIZE (1 << ALIGN_BITS)
+#define ALIGN_MASK (ALIGN_SIZE - 1)
 
 // LZMA remembers the four most recent match distances. Reusing these distances
 // tends to take less space than re-encoding the actual distance value.
-#define REP_DISTANCES 4
+#define REPS 4
 
 #endif

src/liblzma/lzma/lzma_decoder.c

@@ -193,15 +193,15 @@ struct lzma_coder_s {
 	/// Probability tree for the highest two bits of the match distance.
 	/// There is a separate probability tree for match lengths of
 	/// 2 (i.e. MATCH_LEN_MIN), 3, 4, and [5, 273].
-	probability pos_slot[LEN_TO_POS_STATES][POS_SLOTS];
+	probability dist_slot[DIST_STATES][DIST_SLOTS];
 
 	/// Probability trees for additional bits for match distance when the
 	/// distance is in the range [4, 127].
-	probability pos_special[FULL_DISTANCES - END_POS_MODEL_INDEX];
+	probability pos_special[FULL_DISTANCES - DIST_MODEL_END];
 
 	/// Probability tree for the lowest four bits of a match distance
 	/// that is equal to or greater than 128.
-	probability pos_align[ALIGN_TABLE_SIZE];
+	probability pos_align[ALIGN_SIZE];
 
 	/// Length of a normal match
 	lzma_length_decoder match_len_decoder;
@@ -245,8 +245,8 @@ struct lzma_coder_s {
 		SEQ_LITERAL_WRITE,
 		SEQ_IS_REP,
 		seq_len(SEQ_MATCH_LEN),
-		seq_6(SEQ_POS_SLOT),
-		SEQ_POS_MODEL,
+		seq_6(SEQ_DIST_SLOT),
+		SEQ_DIST_MODEL,
 		SEQ_DIRECT,
 		seq_4(SEQ_ALIGN),
 		SEQ_EOPM,
@@ -502,28 +502,28 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 
 			// Prepare to decode the highest two bits of the
 			// match distance.
-			probs = coder->pos_slot[get_len_to_pos_state(len)];
+			probs = coder->dist_slot[get_dist_state(len)];
 			symbol = 1;
 
 #ifdef HAVE_SMALL
-	case SEQ_POS_SLOT:
+	case SEQ_DIST_SLOT:
 			do {
-				rc_bit(probs[symbol], , , SEQ_POS_SLOT);
-			} while (symbol < POS_SLOTS);
+				rc_bit(probs[symbol], , , SEQ_DIST_SLOT);
+			} while (symbol < DIST_SLOTS);
 #else
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT0);
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT1);
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT2);
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT3);
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT4);
-			rc_bit_case(probs[symbol], , , SEQ_POS_SLOT5);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT0);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT1);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT2);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT3);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT4);
+			rc_bit_case(probs[symbol], , , SEQ_DIST_SLOT5);
 #endif
 			// Get rid of the highest bit that was needed for
 			// indexing of the probability array.
-			symbol -= POS_SLOTS;
+			symbol -= DIST_SLOTS;
 			assert(symbol <= 63);
 
-			if (symbol < START_POS_MODEL_INDEX) {
+			if (symbol < DIST_MODEL_START) {
 				// Match distances [0, 3] have only two bits.
 				rep0 = symbol;
 			} else {
@@ -533,7 +533,7 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 				assert(limit >= 1 && limit <= 30);
 				rep0 = 2 + (symbol & 1);
 
-				if (symbol < END_POS_MODEL_INDEX) {
+				if (symbol < DIST_MODEL_END) {
 					// Prepare to decode the low bits for
 					// a distance of [4, 127].
 					assert(limit <= 5);
@@ -553,12 +553,12 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 							- symbol - 1;
 					symbol = 1;
 					offset = 0;
-	case SEQ_POS_MODEL:
+	case SEQ_DIST_MODEL:
 #ifdef HAVE_SMALL
 					do {
 						rc_bit(probs[symbol], ,
 							rep0 += 1 << offset,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 					} while (++offset < limit);
 #else
 					switch (limit) {
@@ -566,25 +566,25 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 						assert(offset == 0);
 						rc_bit(probs[symbol], ,
 							rep0 += 1,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 						++offset;
 						--limit;
 					case 4:
 						rc_bit(probs[symbol], ,
 							rep0 += 1 << offset,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 						++offset;
 						--limit;
 					case 3:
 						rc_bit(probs[symbol], ,
 							rep0 += 1 << offset,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 						++offset;
 						--limit;
 					case 2:
 						rc_bit(probs[symbol], ,
 							rep0 += 1 << offset,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 						++offset;
 						--limit;
 					case 1:
@@ -596,7 +596,7 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 						// "symbol".
 						rc_bit_last(probs[symbol], ,
 							rep0 += 1 << offset,
-							SEQ_POS_MODEL);
+							SEQ_DIST_MODEL);
 					}
 #endif
 				} else {
@@ -637,7 +637,7 @@ lzma_decode(lzma_coder *restrict coder, lzma_dict *restrict dictptr,
 					rc_bit(coder->pos_align[symbol], ,
 							rep0 += 4, SEQ_ALIGN2);
 	case SEQ_ALIGN3:
-					// Like in SEQ_POS_MODEL, we don't
+					// Like in SEQ_DIST_MODEL, we don't
 					// need "symbol" for anything else
 					// than indexing the probability array.
 					rc_bit_last(coder->pos_align[symbol], ,
@@ -891,10 +891,10 @@ lzma_decoder_reset(lzma_coder *coder, const void *opt)
 		bit_reset(coder->is_rep2[i]);
 	}
 
-	for (uint32_t i = 0; i < LEN_TO_POS_STATES; ++i)
-		bittree_reset(coder->pos_slot[i], POS_SLOT_BITS);
+	for (uint32_t i = 0; i < DIST_STATES; ++i)
+		bittree_reset(coder->dist_slot[i], DIST_SLOT_BITS);
 
-	for (uint32_t i = 0; i < FULL_DISTANCES - END_POS_MODEL_INDEX; ++i)
+	for (uint32_t i = 0; i < FULL_DISTANCES - DIST_MODEL_END; ++i)
 		bit_reset(coder->pos_special[i]);
 
 	bittree_reset(coder->pos_align, ALIGN_BITS);

src/liblzma/lzma/lzma_encoder.c

@@ -148,28 +148,28 @@ match(lzma_coder *coder, const uint32_t pos_state,
 	length(&coder->rc, &coder->match_len_encoder, pos_state, len,
 			coder->fast_mode);
 
-	const uint32_t pos_slot = get_pos_slot(distance);
-	const uint32_t len_to_pos_state = get_len_to_pos_state(len);
-	rc_bittree(&coder->rc, coder->pos_slot[len_to_pos_state],
-			POS_SLOT_BITS, pos_slot);
+	const uint32_t dist_slot = get_dist_slot(distance);
+	const uint32_t dist_state = get_dist_state(len);
+	rc_bittree(&coder->rc, coder->dist_slot[dist_state],
+			DIST_SLOT_BITS, dist_slot);
 
-	if (pos_slot >= START_POS_MODEL_INDEX) {
-		const uint32_t footer_bits = (pos_slot >> 1) - 1;
-		const uint32_t base = (2 | (pos_slot & 1)) << footer_bits;
-		const uint32_t pos_reduced = distance - base;
+	if (dist_slot >= DIST_MODEL_START) {
+		const uint32_t footer_bits = (dist_slot >> 1) - 1;
+		const uint32_t base = (2 | (dist_slot & 1)) << footer_bits;
+		const uint32_t dist_reduced = distance - base;
 
-		if (pos_slot < END_POS_MODEL_INDEX) {
-			// Careful here: base - pos_slot - 1 can be -1, but
+		if (dist_slot < DIST_MODEL_END) {
+			// Careful here: base - dist_slot - 1 can be -1, but
 			// rc_bittree_reverse starts at probs[1], not probs[0].
 			rc_bittree_reverse(&coder->rc,
-				coder->pos_special + base - pos_slot - 1,
-				footer_bits, pos_reduced);
+				coder->dist_special + base - dist_slot - 1,
+				footer_bits, dist_reduced);
 		} else {
-			rc_direct(&coder->rc, pos_reduced >> ALIGN_BITS,
+			rc_direct(&coder->rc, dist_reduced >> ALIGN_BITS,
 					footer_bits - ALIGN_BITS);
 			rc_bittree_reverse(
-					&coder->rc, coder->pos_align,
-					ALIGN_BITS, pos_reduced & ALIGN_MASK);
+					&coder->rc, coder->dist_align,
+					ALIGN_BITS, dist_reduced & ALIGN_MASK);
 			++coder->align_price_count;
 		}
 	}
@@ -247,7 +247,7 @@ encode_symbol(lzma_coder *coder, lzma_mf *mf,
 		rc_bit(&coder->rc,
 			&coder->is_match[coder->state][pos_state], 1);
 
-		if (back < REP_DISTANCES) {
+		if (back < REPS) {
 			// It's a repeated match i.e. the same distance
 			// has been used earlier.
 			rc_bit(&coder->rc, &coder->is_rep[coder->state], 1);
@@ -255,7 +255,7 @@ encode_symbol(lzma_coder *coder, lzma_mf *mf,
 		} else {
 			// Normal match
 			rc_bit(&coder->rc, &coder->is_rep[coder->state], 0);
-			match(coder, pos_state, back - REP_DISTANCES, len);
+			match(coder, pos_state, back - REPS, len);
 		}
 	}
 
@@ -353,9 +353,9 @@ lzma_lzma_encode(lzma_coder *restrict coder, lzma_mf *restrict mf,
 
 		// Get optimal match (repeat position and length).
 		// Value ranges for pos:
-		//   - [0, REP_DISTANCES): repeated match
-		//   - [REP_DISTANCES, UINT32_MAX):
-		//     match at (pos - REP_DISTANCES)
+		//   - [0, REPS): repeated match
+		//   - [REPS, UINT32_MAX):
+		//     match at (pos - REPS)
 		//   - UINT32_MAX: not a match but a literal
 		// Value ranges for len:
 		//   - [MATCH_LEN_MIN, MATCH_LEN_MAX]
@@ -487,7 +487,7 @@ lzma_lzma_encoder_reset(lzma_coder *coder, const lzma_options_lzma *options)
 
 	// State
 	coder->state = STATE_LIT_LIT;
-	for (size_t i = 0; i < REP_DISTANCES; ++i)
+	for (size_t i = 0; i < REPS; ++i)
 		coder->reps[i] = 0;
 
 	literal_init(coder->literal, options->lc, options->lp);
@@ -505,14 +505,14 @@ lzma_lzma_encoder_reset(lzma_coder *coder, const lzma_options_lzma *options)
 		bit_reset(coder->is_rep2[i]);
 	}
 
-	for (size_t i = 0; i < FULL_DISTANCES - END_POS_MODEL_INDEX; ++i)
-		bit_reset(coder->pos_special[i]);
+	for (size_t i = 0; i < FULL_DISTANCES - DIST_MODEL_END; ++i)
+		bit_reset(coder->dist_special[i]);
 
 	// Bit tree encoders
-	for (size_t i = 0; i < LEN_TO_POS_STATES; ++i)
-		bittree_reset(coder->pos_slot[i], POS_SLOT_BITS);
+	for (size_t i = 0; i < DIST_STATES; ++i)
+		bittree_reset(coder->dist_slot[i], DIST_SLOT_BITS);
 
-	bittree_reset(coder->pos_align, ALIGN_BITS);
+	bittree_reset(coder->dist_align, ALIGN_BITS);
 
 	// Length encoders
 	length_encoder_reset(&coder->match_len_encoder,

src/liblzma/lzma/lzma_encoder_optimum_fast.c

@@ -46,7 +46,7 @@ lzma_lzma_optimum_fast(lzma_coder *restrict coder, lzma_mf *restrict mf,
 	uint32_t rep_len = 0;
 	uint32_t rep_index = 0;
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i) {
+	for (uint32_t i = 0; i < REPS; ++i) {
 		// Pointer to the beginning of the match candidate
 		const uint8_t *const buf_back = buf - coder->reps[i] - 1;
 
@@ -79,8 +79,7 @@ lzma_lzma_optimum_fast(lzma_coder *restrict coder, lzma_mf *restrict mf,
 	// We didn't find a long enough repeated match. Encode it as a normal
 	// match if the match length is at least nice_len.
 	if (len_main >= nice_len) {
-		*back_res = coder->matches[matches_count - 1].dist
-				+ REP_DISTANCES;
+		*back_res = coder->matches[matches_count - 1].dist + REPS;
 		*len_res = len_main;
 		mf_skip(mf, len_main - 1);
 		return;
@@ -155,7 +154,7 @@ lzma_lzma_optimum_fast(lzma_coder *restrict coder, lzma_mf *restrict mf,
 
 	const uint32_t limit = len_main - 1;
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i) {
+	for (uint32_t i = 0; i < REPS; ++i) {
 		const uint8_t *const buf_back = buf - coder->reps[i] - 1;
 
 		if (not_equal_16(buf, buf_back))
@@ -172,7 +171,7 @@ lzma_lzma_optimum_fast(lzma_coder *restrict coder, lzma_mf *restrict mf,
 		}
 	}
 
-	*back_res = back_main + REP_DISTANCES;
+	*back_res = back_main + REPS;
 	*len_res = len_main;
 	mf_skip(mf, len_main - 2);
 	return;

src/liblzma/lzma/lzma_encoder_optimum_normal.c

@@ -108,18 +108,18 @@ get_rep_price(const lzma_coder *const coder, const uint32_t rep_index,
 
 
 static inline uint32_t
-get_pos_len_price(const lzma_coder *const coder, const uint32_t pos,
+get_dist_len_price(const lzma_coder *const coder, const uint32_t dist,
 		const uint32_t len, const uint32_t pos_state)
 {
-	const uint32_t len_to_pos_state = get_len_to_pos_state(len);
+	const uint32_t dist_state = get_dist_state(len);
 	uint32_t price;
 
-	if (pos < FULL_DISTANCES) {
-		price = coder->distances_prices[len_to_pos_state][pos];
+	if (dist < FULL_DISTANCES) {
+		price = coder->dist_prices[dist_state][dist];
 	} else {
-		const uint32_t pos_slot = get_pos_slot_2(pos);
-		price = coder->pos_slot_prices[len_to_pos_state][pos_slot]
-				+ coder->align_prices[pos & ALIGN_MASK];
+		const uint32_t dist_slot = get_dist_slot_2(dist);
+		price = coder->dist_slot_prices[dist_state][dist_slot]
+				+ coder->align_prices[dist & ALIGN_MASK];
 	}
 
 	price += get_len_price(&coder->match_len_encoder, len, pos_state);
@@ -129,55 +129,53 @@ get_pos_len_price(const lzma_coder *const coder, const uint32_t pos,
 
 
 static void
-fill_distances_prices(lzma_coder *coder)
+fill_dist_prices(lzma_coder *coder)
 {
-	for (uint32_t len_to_pos_state = 0;
-			len_to_pos_state < LEN_TO_POS_STATES;
-			++len_to_pos_state) {
+	for (uint32_t dist_state = 0; dist_state < DIST_STATES; ++dist_state) {
 
-		uint32_t *const pos_slot_prices
-				= coder->pos_slot_prices[len_to_pos_state];
+		uint32_t *const dist_slot_prices
+				= coder->dist_slot_prices[dist_state];
 
-		// Price to encode the pos_slot.
-		for (uint32_t pos_slot = 0;
-				pos_slot < coder->dist_table_size; ++pos_slot)
-			pos_slot_prices[pos_slot] = rc_bittree_price(
-					coder->pos_slot[len_to_pos_state],
-					POS_SLOT_BITS, pos_slot);
+		// Price to encode the dist_slot.
+		for (uint32_t dist_slot = 0;
+				dist_slot < coder->dist_table_size; ++dist_slot)
+			dist_slot_prices[dist_slot] = rc_bittree_price(
+					coder->dist_slot[dist_state],
+					DIST_SLOT_BITS, dist_slot);
 
 		// For matches with distance >= FULL_DISTANCES, add the price
 		// of the direct bits part of the match distance. (Align bits
 		// are handled by fill_align_prices()).
-		for (uint32_t pos_slot = END_POS_MODEL_INDEX;
-				pos_slot < coder->dist_table_size; ++pos_slot)
-			pos_slot_prices[pos_slot] += rc_direct_price(
-					((pos_slot >> 1) - 1) - ALIGN_BITS);
+		for (uint32_t dist_slot = DIST_MODEL_END;
+				dist_slot < coder->dist_table_size;
+				++dist_slot)
+			dist_slot_prices[dist_slot] += rc_direct_price(
+					((dist_slot >> 1) - 1) - ALIGN_BITS);
 
 		// Distances in the range [0, 3] are fully encoded with
-		// pos_slot, so they are used for coder->distances_prices
+		// dist_slot, so they are used for coder->dist_prices
 		// as is.
-		for (uint32_t i = 0; i < START_POS_MODEL_INDEX; ++i)
-			coder->distances_prices[len_to_pos_state][i]
-					= pos_slot_prices[i];
+		for (uint32_t i = 0; i < DIST_MODEL_START; ++i)
+			coder->dist_prices[dist_state][i]
+					= dist_slot_prices[i];
 	}
 
-	// Distances in the range [4, 127] depend on pos_slot and pos_special.
-	// We do this in a loop separate from the above loop to avoid
-	// redundant calls to get_pos_slot().
-	for (uint32_t i = START_POS_MODEL_INDEX; i < FULL_DISTANCES; ++i) {
-		const uint32_t pos_slot = get_pos_slot(i);
-		const uint32_t footer_bits = ((pos_slot >> 1) - 1);
-		const uint32_t base = (2 | (pos_slot & 1)) << footer_bits;
+	// Distances in the range [4, 127] depend on dist_slot and
+	// dist_special. We do this in a loop separate from the above
+	// loop to avoid redundant calls to get_dist_slot().
+	for (uint32_t i = DIST_MODEL_START; i < FULL_DISTANCES; ++i) {
+		const uint32_t dist_slot = get_dist_slot(i);
+		const uint32_t footer_bits = ((dist_slot >> 1) - 1);
+		const uint32_t base = (2 | (dist_slot & 1)) << footer_bits;
 		const uint32_t price = rc_bittree_reverse_price(
-				coder->pos_special + base - pos_slot - 1,
+				coder->dist_special + base - dist_slot - 1,
 				footer_bits, i - base);
 
-		for (uint32_t len_to_pos_state = 0;
-				len_to_pos_state < LEN_TO_POS_STATES;
-				++len_to_pos_state)
-			coder->distances_prices[len_to_pos_state][i]
-					= price + coder->pos_slot_prices[
-						len_to_pos_state][pos_slot];
+		for (uint32_t dist_state = 0; dist_state < DIST_STATES;
+				++dist_state)
+			coder->dist_prices[dist_state][i]
+					= price + coder->dist_slot_prices[
+						dist_state][dist_slot];
 	}
 
 	coder->match_price_count = 0;
@@ -188,9 +186,9 @@ fill_distances_prices(lzma_coder *coder)
 static void
 fill_align_prices(lzma_coder *coder)
 {
-	for (uint32_t i = 0; i < ALIGN_TABLE_SIZE; ++i)
+	for (uint32_t i = 0; i < ALIGN_SIZE; ++i)
 		coder->align_prices[i] = rc_bittree_reverse_price(
-				coder->pos_align, ALIGN_BITS, i);
+				coder->dist_align, ALIGN_BITS, i);
 
 	coder->align_price_count = 0;
 	return;
@@ -296,10 +294,10 @@ helper1(lzma_coder *restrict coder, lzma_mf *restrict mf,
 
 	const uint8_t *const buf = mf_ptr(mf) - 1;
 
-	uint32_t rep_lens[REP_DISTANCES];
+	uint32_t rep_lens[REPS];
 	uint32_t rep_max_index = 0;
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i) {
+	for (uint32_t i = 0; i < REPS; ++i) {
 		const uint8_t *const buf_back = buf - coder->reps[i] - 1;
 
 		if (not_equal_16(buf, buf_back)) {
@@ -326,8 +324,7 @@ helper1(lzma_coder *restrict coder, lzma_mf *restrict mf,
 
 
 	if (len_main >= nice_len) {
-		*back_res = coder->matches[matches_count - 1].dist
-				+ REP_DISTANCES;
+		*back_res = coder->matches[matches_count - 1].dist + REPS;
 		*len_res = len_main;
 		mf_skip(mf, len_main - 1);
 		return UINT32_MAX;
@@ -381,7 +378,7 @@ helper1(lzma_coder *restrict coder, lzma_mf *restrict mf,
 
 	coder->opts[1].pos_prev = 0;
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i)
+	for (uint32_t i = 0; i < REPS; ++i)
 		coder->opts[0].backs[i] = coder->reps[i];
 
 	uint32_t len = len_end;
@@ -390,7 +387,7 @@ helper1(lzma_coder *restrict coder, lzma_mf *restrict mf,
 	} while (--len >= 2);
 
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i) {
+	for (uint32_t i = 0; i < REPS; ++i) {
 		uint32_t rep_len = rep_lens[i];
 		if (rep_len < 2)
 			continue;
@@ -426,14 +423,13 @@ helper1(lzma_coder *restrict coder, lzma_mf *restrict mf,
 		for(; ; ++len) {
 			const uint32_t dist = coder->matches[i].dist;
 			const uint32_t cur_and_len_price = normal_match_price
-					+ get_pos_len_price(coder,
+					+ get_dist_len_price(coder,
 						dist, len, pos_state);
 
 			if (cur_and_len_price < coder->opts[len].price) {
 				coder->opts[len].price = cur_and_len_price;
 				coder->opts[len].pos_prev = 0;
-				coder->opts[len].back_prev
-						= dist + REP_DISTANCES;
+				coder->opts[len].back_prev = dist + REPS;
 				coder->opts[len].prev_1_is_literal = false;
 			}
 
@@ -463,7 +459,7 @@ helper2(lzma_coder *coder, uint32_t *reps, const uint8_t *buf,
 		if (coder->opts[cur].prev_2) {
 			state = coder->opts[coder->opts[cur].pos_prev_2].state;
 
-			if (coder->opts[cur].back_prev_2 < REP_DISTANCES)
+			if (coder->opts[cur].back_prev_2 < REPS)
 				update_long_rep(state);
 			else
 				update_match(state);
@@ -492,33 +488,33 @@ helper2(lzma_coder *coder, uint32_t *reps, const uint8_t *buf,
 			update_long_rep(state);
 		} else {
 			pos = coder->opts[cur].back_prev;
-			if (pos < REP_DISTANCES)
+			if (pos < REPS)
 				update_long_rep(state);
 			else
 				update_match(state);
 		}
 
-		if (pos < REP_DISTANCES) {
+		if (pos < REPS) {
 			reps[0] = coder->opts[pos_prev].backs[pos];
 
 			uint32_t i;
 			for (i = 1; i <= pos; ++i)
 				reps[i] = coder->opts[pos_prev].backs[i - 1];
 
-			for (; i < REP_DISTANCES; ++i)
+			for (; i < REPS; ++i)
 				reps[i] = coder->opts[pos_prev].backs[i];
 
 		} else {
-			reps[0] = pos - REP_DISTANCES;
+			reps[0] = pos - REPS;
 
-			for (uint32_t i = 1; i < REP_DISTANCES; ++i)
+			for (uint32_t i = 1; i < REPS; ++i)
 				reps[i] = coder->opts[pos_prev].backs[i - 1];
 		}
 	}
 
 	coder->opts[cur].state = state;
 
-	for (uint32_t i = 0; i < REP_DISTANCES; ++i)
+	for (uint32_t i = 0; i < REPS; ++i)
 		coder->opts[cur].backs[i] = reps[i];
 
 	const uint32_t cur_price = coder->opts[cur].price;
@@ -611,7 +607,7 @@ helper2(lzma_coder *coder, uint32_t *reps, const uint8_t *buf,
 
 	uint32_t start_len = 2; // speed optimization
 
-	for (uint32_t rep_index = 0; rep_index < REP_DISTANCES; ++rep_index) {
+	for (uint32_t rep_index = 0; rep_index < REPS; ++rep_index) {
 		const uint8_t *const buf_back = buf - reps[rep_index] - 1;
 		if (not_equal_16(buf, buf_back))
 			continue;
@@ -728,14 +724,14 @@ helper2(lzma_coder *coder, uint32_t *reps, const uint8_t *buf,
 		for (uint32_t len_test = start_len; ; ++len_test) {
 			const uint32_t cur_back = coder->matches[i].dist;
 			uint32_t cur_and_len_price = normal_match_price
-					+ get_pos_len_price(coder,
+					+ get_dist_len_price(coder,
 						cur_back, len_test, pos_state);
 
 			if (cur_and_len_price < coder->opts[cur + len_test].price) {
 				coder->opts[cur + len_test].price = cur_and_len_price;
 				coder->opts[cur + len_test].pos_prev = cur;
 				coder->opts[cur + len_test].back_prev
-						= cur_back + REP_DISTANCES;
+						= cur_back + REPS;
 				coder->opts[cur + len_test].prev_1_is_literal = false;
 			}
 
@@ -795,7 +791,7 @@ helper2(lzma_coder *coder, uint32_t *reps, const uint8_t *buf,
 						coder->opts[offset].prev_2 = true;
 						coder->opts[offset].pos_prev_2 = cur;
 						coder->opts[offset].back_prev_2
-								= cur_back + REP_DISTANCES;
+								= cur_back + REPS;
 					}
 					//}
 				}
@@ -831,9 +827,9 @@ lzma_lzma_optimum_normal(lzma_coder *restrict coder, lzma_mf *restrict mf,
 	// In liblzma they were moved into this single place.
 	if (mf->read_ahead == 0) {
 		if (coder->match_price_count >= (1 << 7))
-			fill_distances_prices(coder);
+			fill_dist_prices(coder);
 
-		if (coder->align_price_count >= ALIGN_TABLE_SIZE)
+		if (coder->align_price_count >= ALIGN_SIZE)
 			fill_align_prices(coder);
 	}
 
@@ -845,7 +841,7 @@ lzma_lzma_optimum_normal(lzma_coder *restrict coder, lzma_mf *restrict mf,
 	if (len_end == UINT32_MAX)
 		return;
 
-	uint32_t reps[REP_DISTANCES];
+	uint32_t reps[REPS];
 	memcpy(reps, coder->reps, sizeof(reps));
 
 	uint32_t cur;

src/liblzma/lzma/lzma_encoder_private.h

@@ -64,7 +64,7 @@ typedef struct {
 	uint32_t pos_prev;  // pos_next;
 	uint32_t back_prev;
 
-	uint32_t backs[REP_DISTANCES];
+	uint32_t backs[REPS];
 
 } lzma_optimal;
 
@@ -77,7 +77,7 @@ struct lzma_coder_s {
 	lzma_lzma_state state;
 
 	/// The four most recent match distances
-	uint32_t reps[REP_DISTANCES];
+	uint32_t reps[REPS];
 
 	/// Array of match candidates
 	lzma_match matches[MATCH_LEN_MAX + 1];
@@ -112,9 +112,9 @@ struct lzma_coder_s {
 	probability is_rep1[STATES];
 	probability is_rep2[STATES];
 	probability is_rep0_long[STATES][POS_STATES_MAX];
-	probability pos_slot[LEN_TO_POS_STATES][POS_SLOTS];
-	probability pos_special[FULL_DISTANCES - END_POS_MODEL_INDEX];
-	probability pos_align[ALIGN_TABLE_SIZE];
+	probability dist_slot[DIST_STATES][DIST_SLOTS];
+	probability dist_special[FULL_DISTANCES - DIST_MODEL_END];
+	probability dist_align[ALIGN_SIZE];
 
 	// These are the same as in lzma_decoder.c except that the encoders
 	// include also price tables.
@@ -122,12 +122,12 @@ struct lzma_coder_s {
 	lzma_length_encoder rep_len_encoder;
 
 	// Price tables
-	uint32_t pos_slot_prices[LEN_TO_POS_STATES][POS_SLOTS];
-	uint32_t distances_prices[LEN_TO_POS_STATES][FULL_DISTANCES];
+	uint32_t dist_slot_prices[DIST_STATES][DIST_SLOTS];
+	uint32_t dist_prices[DIST_STATES][FULL_DISTANCES];
 	uint32_t dist_table_size;
 	uint32_t match_price_count;
 
-	uint32_t align_prices[ALIGN_TABLE_SIZE];
+	uint32_t align_prices[ALIGN_SIZE];
 	uint32_t align_price_count;
 
 	// Optimal

src/scripts/xzdiff.1

@@ -6,7 +6,7 @@
 .\"
 .\" License: GNU GPLv2+
 .\"
-.TH XZDIFF 1 "2010-09-27" "Tukaani" "XZ Utils"
+.TH XZDIFF 1 "2011-03-19" "Tukaani" "XZ Utils"
 .SH NAME
 xzcmp, xzdiff, lzcmp, lzdiff \- compare compressed files
 .SH SYNOPSIS
@@ -33,8 +33,9 @@ on files compressed with
 .BR xz (1),
 .BR lzma (1),
 .BR gzip (1),
+.BR bzip2 (1),
 or
-.BR bzip2 (1).
+.BR lzop (1).
 All options specified are passed directly to
 .BR cmp (1)
 or
@@ -66,6 +67,7 @@ are provided for backward compatibility with LZMA Utils.
 .BR xz (1),
 .BR gzip (1),
 .BR bzip2 (1),
+.BR lzop (1),
 .BR zdiff (1)
 .SH BUGS
 Messages from the

src/scripts/xzdiff.in

@@ -19,10 +19,10 @@
 
 # Instead of unsetting XZ_OPT, just make sure that xz will use file format
 # autodetection. This way memory usage limit and thread limit can be
-# specified via XZ_OPT. With gzip and bzip2 it's OK to just unset the
+# specified via XZ_OPT. With gzip, bzip2, and lzop it's OK to just unset the
 # environment variables.
 xz='@xz@ --format=auto'
-unset GZIP BZIP BZIP2
+unset GZIP BZIP BZIP2 LZOP
 
 case ${0##*/} in
   *cmp*) prog=xzcmp; cmp=${CMP:-cmp};;
@@ -77,17 +77,21 @@ if test $# -eq 1; then
       xz1=bzip2;;
     *[-.][zZ] | *_z | *[-.]gz | *.t[ag]z)
       xz1=gzip;;
+    *[-.]lzo | *.tzo)
+      xz1=lzop;;
     *)
       echo >&2 "$0: $1: Unknown compressed file name suffix"
       exit 2;;
   esac
   case $1 in
-    *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma)
-      FILE=`expr "X$1" : 'X\(.*\)[-.][abglmxzZ2]*$'`;;
+    *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *[-.]lzo)
+      FILE=`expr "X$1" : 'X\(.*\)[-.][abglmoxzZ2]*$'`;;
     *.t[abglx]z)
       FILE=`expr "X$1" : 'X\(.*[-.]t\)[abglx]z$'`ar;;
     *.tbz2)
       FILE=`expr "X$1" : 'X\(.*[-.]t\)bz2$'`ar;;
+    *.tzo)
+      FILE=`expr "X$1" : 'X\(.*[-.]t\)zo$'`ar;;
   esac
   xz_status=$(
     exec 4>&1
@@ -97,15 +101,17 @@ elif test $# -eq 2; then
   case $1 in
     *[-.]bz2 | *.tbz | *.tbz2) xz1=bzip2;;
     *[-.][zZ] | *_z | *[-.]gz | *.t[ag]z) xz1=gzip;;
+    *[-.]lzo | *.tzo) xz1=lzop;;
   esac
   case $2 in
     *[-.]bz2 | *.tbz | *.tbz2) xz2=bzip2;;
     *[-.][zZ] | *_z | *[-.]gz | *.t[ag]z) xz2=gzip;;
+    *[-.]lzo | *.tzo) xz2=lzop;;
   esac
   case $1 in
-    *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | -)
+    *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | *[-.]lzo | *.tzo | -)
       case "$2" in
-        *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | -)
+        *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | *[-.]lzo | *.tzo | -)
           if test "$1$2" = --; then
             xz_status=$(
               exec 4>&1
@@ -125,7 +131,7 @@ elif test $# -eq 2; then
               *) xz_status=0;;
             esac
           else
-            F=`expr "/$2" : '.*/\(.*\)[-.][ablmtxz2]*$'` || F=$prog
+            F=`expr "/$2" : '.*/\(.*\)[-.][ablmotxz2]*$'` || F=$prog
             tmp=
             trap '
               test -n "$tmp" && rm -f "$tmp"
@@ -152,7 +158,7 @@ elif test $# -eq 2; then
     esac;;
   *)
     case "$2" in
-      *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | -)
+      *[-.][zZ] | *_z | *[-.][gx]z | *[-.]bz2 | *[-.]lzma | *.t[abglx]z | *.tbz2 | *[-.]lzo | *.tzo | -)
         xz_status=$(
           exec 4>&1
           ($xz2 -cdfq -- "$2" 4>&-; echo $? >&4) 3>&- |

src/scripts/xzgrep.1

@@ -6,7 +6,7 @@
 .\"
 .\" License: GNU GPLv2+
 .\"
-.TH XZGREP 1 "2010-09-27" "Tukaani" "XZ Utils"
+.TH XZGREP 1 "2011-03-19" "Tukaani" "XZ Utils"
 .SH NAME
 xzgrep \- search compressed files for a regular expression
 .SH SYNOPSIS
@@ -40,8 +40,9 @@ which may be either uncompressed or compressed with
 .BR xz (1),
 .BR lzma (1),
 .BR gzip (1),
+.BR bzip2 (1),
 or
-.BR bzip2 (1).
+.BR lzop (1).
 All options specified are passed directly to
 .BR grep (1).
 .PP
@@ -51,9 +52,10 @@ is specified, then standard input is decompressed if necessary
 and fed to
 .BR grep (1).
 When reading from standard input,
-.BR gzip (1)
+.BR gzip (1),
+.BR bzip2 (1),
 and
-.BR bzip2 (1)
+.BR lzop (1)
 compressed files are not supported.
 .PP
 If
@@ -92,4 +94,5 @@ or
 .BR xz (1),
 .BR gzip (1),
 .BR bzip2 (1),
+.BR lzop (1),
 .BR zgrep (1)

src/scripts/xzgrep.in

@@ -22,10 +22,10 @@
 
 # Instead of unsetting XZ_OPT, just make sure that xz will use file format
 # autodetection. This way memory usage limit and thread limit can be
-# specified via XZ_OPT. With gzip and bzip2 it's OK to just unset the
+# specified via XZ_OPT. With gzip, bzip2, and lzop it's OK to just unset the
 # environment variables.
 xz='@xz@ --format=auto'
-unset GZIP BZIP BZIP2
+unset GZIP BZIP BZIP2 LZOP
 
 case ${0##/*} in
   *egrep*) prog=xzegrep; grep=${GREP:-egrep};;
@@ -126,6 +126,10 @@ while test $# -ne 0; do
   grep="$grep $option$optarg"
 done
 
+if test $files_with_matches -eq 1 || test $files_without_matches -eq 1; then
+  grep="$grep -q"
+fi
+
 eval "set -- $operands "'${1+"$@"}'
 
 if test $have_pat -eq 0; then
@@ -149,6 +153,7 @@ for i; do
   case $i in
     *[-.][zZ] | *_z | *[-.]gz | *.t[ag]z) uncompress="gzip -cdfq";;
     *[-.]bz2 | *[-.]tbz | *.tbz2) uncompress="bzip2 -cdfq";;
+    *[-.]lzo | *[-.]tzo) uncompress="lzop -cdfq";;
     *) uncompress="$xz -cdfq";;
   esac
   # Fail if xz or grep (or sed) fails.
@@ -156,9 +161,9 @@ for i; do
     exec 5>&1
     ($uncompress -- "$i" 5>&-; echo $? >&5) 3>&- |
     if test $files_with_matches -eq 1; then
-      eval "$grep" -q && { printf '%s\n' "$i" || exit 2; }
+      eval "$grep" && { printf '%s\n' "$i" || exit 2; }
     elif test $files_without_matches -eq 1; then
-      eval "$grep" -q || {
+      eval "$grep" || {
         r=$?
         if test $r -eq 1; then
           printf '%s\n' "$i" || r=2

src/xz/args.c

@@ -68,9 +68,11 @@ parse_real(args_info *args, int argc, char **argv)
 		OPT_LZMA1,
 		OPT_LZMA2,
 
+		OPT_SINGLE_STREAM,
 		OPT_NO_SPARSE,
 		OPT_FILES,
 		OPT_FILES0,
+		OPT_BLOCK_SIZE,
 		OPT_MEM_COMPRESS,
 		OPT_MEM_DECOMPRESS,
 		OPT_NO_ADJUST,
@@ -94,6 +96,7 @@ parse_real(args_info *args, int argc, char **argv)
 		{ "force",        no_argument,       NULL,  'f' },
 		{ "stdout",       no_argument,       NULL,  'c' },
 		{ "to-stdout",    no_argument,       NULL,  'c' },
+		{ "single-stream", no_argument,      NULL,  OPT_SINGLE_STREAM },
 		{ "no-sparse",    no_argument,       NULL,  OPT_NO_SPARSE },
 		{ "suffix",       required_argument, NULL,  'S' },
 		// { "recursive",      no_argument,       NULL,  'r' }, // TODO
@@ -103,6 +106,7 @@ parse_real(args_info *args, int argc, char **argv)
 		// Basic compression settings
 		{ "format",       required_argument, NULL,  'F' },
 		{ "check",        required_argument, NULL,  'C' },
+		{ "block-size",   required_argument, NULL,  OPT_BLOCK_SIZE },
 		{ "memlimit-compress",   required_argument, NULL, OPT_MEM_COMPRESS },
 		{ "memlimit-decompress", required_argument, NULL, OPT_MEM_DECOMPRESS },
 		{ "memlimit",     required_argument, NULL,  'M' },
@@ -175,8 +179,9 @@ parse_real(args_info *args, int argc, char **argv)
 			break;
 
 		case 'T':
-			hardware_threadlimit_set(str_to_uint64(
-					"threads", optarg, 0, UINT32_MAX));
+			// The max is from src/liblzma/common/common.h.
+			hardware_threads_set(str_to_uint64("threads",
+					optarg, 0, 16384));
 			break;
 
 		// --version
@@ -368,6 +373,15 @@ parse_real(args_info *args, int argc, char **argv)
 			break;
 		}
 
+		case OPT_BLOCK_SIZE:
+			opt_block_size = str_to_uint64("block-size", optarg,
+					0, LZMA_VLI_MAX);
+			break;
+
+		case OPT_SINGLE_STREAM:
+			opt_single_stream = true;
+			break;
+
 		case OPT_NO_SPARSE:
 			io_no_sparse();
 			break;

src/xz/coder.c

@@ -24,6 +24,8 @@ enum coder_init_ret {
 enum operation_mode opt_mode = MODE_COMPRESS;
 enum format_type opt_format = FORMAT_AUTO;
 bool opt_auto_adjust = true;
+bool opt_single_stream = false;
+uint64_t opt_block_size = 0;
 
 
 /// Stream used to communicate with liblzma
@@ -37,10 +39,10 @@ static io_buf in_buf;
 static io_buf out_buf;
 
 /// Number of filters. Zero indicates that we are using a preset.
-static size_t filters_count = 0;
+static uint32_t filters_count = 0;
 
 /// Number of the preset (0-9)
-static size_t preset_number = 6;
+static uint32_t preset_number = 6;
 
 /// If a preset is used (no custom filter chain) and preset_extreme is true,
 /// a significantly slower compression is used to achieve slightly better
@@ -53,6 +55,14 @@ static lzma_check check;
 /// This becomes false if the --check=CHECK option is used.
 static bool check_default = true;
 
+#ifdef HAVE_PTHREAD
+static lzma_mt mt_options = {
+	.flags = 0,
+	.timeout = 300,
+	.filters = filters,
+};
+#endif
+
 
 extern void
 coder_set_check(lzma_check new_check)
@@ -64,7 +74,7 @@ coder_set_check(lzma_check new_check)
 
 
 extern void
-coder_set_preset(size_t new_preset)
+coder_set_preset(uint32_t new_preset)
 {
 	preset_number = new_preset;
 
@@ -115,6 +125,15 @@ memlimit_too_small(uint64_t memory_usage)
 extern void
 coder_set_compression_settings(void)
 {
+	// The default check type is CRC64, but fallback to CRC32
+	// if CRC64 isn't supported by the copy of liblzma we are
+	// using. CRC32 is always supported.
+	if (check_default) {
+		check = LZMA_CHECK_CRC64;
+		if (!lzma_check_is_supported(check))
+			check = LZMA_CHECK_CRC32;
+	}
+
 	// Options for LZMA1 or LZMA2 in case we are using a preset.
 	static lzma_options_lzma opt_lzma;
 
@@ -168,15 +187,26 @@ coder_set_compression_settings(void)
 	// Print the selected filter chain.
 	message_filters_show(V_DEBUG, filters);
 
-	// If using --format=raw, we can be decoding. The memusage function
-	// also validates the filter chain and the options used for the
-	// filters.
+	// Get the memory usage. Note that if --format=raw was used,
+	// we can be decompressing.
 	const uint64_t memory_limit = hardware_memlimit_get(opt_mode);
 	uint64_t memory_usage;
-	if (opt_mode == MODE_COMPRESS)
-		memory_usage = lzma_raw_encoder_memusage(filters);
-	else
+	if (opt_mode == MODE_COMPRESS) {
+#ifdef HAVE_PTHREAD
+		if (opt_format == FORMAT_XZ && hardware_threads_get() > 1) {
+			mt_options.threads = hardware_threads_get();
+			mt_options.block_size = opt_block_size;
+			mt_options.check = check;
+			memory_usage = lzma_stream_encoder_mt_memusage(
+					&mt_options);
+		} else
+#endif
+		{
+			memory_usage = lzma_raw_encoder_memusage(filters);
+		}
+	} else {
 		memory_usage = lzma_raw_decoder_memusage(filters);
+	}
 
 	if (memory_usage == UINT64_MAX)
 		message_fatal(_("Unsupported filter chain or filter options"));
@@ -192,90 +222,99 @@ coder_set_compression_settings(void)
 						round_up_to_mib(decmem), 0));
 	}
 
-	if (memory_usage > memory_limit) {
-		// If --no-auto-adjust was used or we didn't find LZMA1 or
-		// LZMA2 as the last filter, give an error immediately.
-		// --format=raw implies --no-auto-adjust.
-		if (!opt_auto_adjust || opt_format == FORMAT_RAW)
-			memlimit_too_small(memory_usage);
+	if (memory_usage <= memory_limit)
+		return;
 
-		assert(opt_mode == MODE_COMPRESS);
+	// If --no-auto-adjust was used or we didn't find LZMA1 or
+	// LZMA2 as the last filter, give an error immediately.
+	// --format=raw implies --no-auto-adjust.
+	if (!opt_auto_adjust || opt_format == FORMAT_RAW)
+		memlimit_too_small(memory_usage);
 
-		// Look for the last filter if it is LZMA2 or LZMA1, so
-		// we can make it use less RAM. With other filters we don't
-		// know what to do.
-		size_t i = 0;
-		while (filters[i].id != LZMA_FILTER_LZMA2
-				&& filters[i].id != LZMA_FILTER_LZMA1) {
-			if (filters[i].id == LZMA_VLI_UNKNOWN)
+	assert(opt_mode == MODE_COMPRESS);
+
+#ifdef HAVE_PTHREAD
+	if (opt_format == FORMAT_XZ && mt_options.threads > 1) {
+		// Try to reduce the number of threads before
+		// adjusting the compression settings down.
+		do {
+			// FIXME? The real single-threaded mode has
+			// lower memory usage, but it's not comparable
+			// because it doesn't write the size info
+			// into Block Headers.
+			if (--mt_options.threads == 0)
 				memlimit_too_small(memory_usage);
 
-			++i;
-		}
-
-		// Decrease the dictionary size until we meet the memory
-		// usage limit. First round down to full mebibytes.
-		lzma_options_lzma *opt = filters[i].options;
-		const uint32_t orig_dict_size = opt->dict_size;
-		opt->dict_size &= ~((UINT32_C(1) << 20) - 1);
-		while (true) {
-			// If it is below 1 MiB, auto-adjusting failed. We
-			// could be more sophisticated and scale it down even
-			// more, but let's see if many complain about this
-			// version.
-			//
-			// FIXME: Displays the scaled memory usage instead
-			// of the original.
-			if (opt->dict_size < (UINT32_C(1) << 20))
-				memlimit_too_small(memory_usage);
-
-			memory_usage = lzma_raw_encoder_memusage(filters);
+			memory_usage = lzma_stream_encoder_mt_memusage(
+					&mt_options);
 			if (memory_usage == UINT64_MAX)
 				message_bug();
 
-			// Accept it if it is low enough.
-			if (memory_usage <= memory_limit)
-				break;
+		} while (memory_usage > memory_limit);
 
-			// Otherwise 1 MiB down and try again. I hope this
-			// isn't too slow method for cases where the original
-			// dict_size is very big.
-			opt->dict_size -= UINT32_C(1) << 20;
-		}
+		message(V_WARNING, _("Adjusted the number of threads "
+			"from %s to %s to not exceed "
+			"the memory usage limit of %s MiB"),
+			uint64_to_str(hardware_threads_get(), 0),
+			uint64_to_str(mt_options.threads, 1),
+			uint64_to_str(round_up_to_mib(
+				memory_limit), 2));
+	}
+#endif
 
-		// Tell the user that we decreased the dictionary size.
-		message(V_WARNING, _("Adjusted LZMA%c dictionary size "
-				"from %s MiB to %s MiB to not exceed "
-				"the memory usage limit of %s MiB"),
-				filters[i].id == LZMA_FILTER_LZMA2
-					? '2' : '1',
-				uint64_to_str(orig_dict_size >> 20, 0),
-				uint64_to_str(opt->dict_size >> 20, 1),
-				uint64_to_str(round_up_to_mib(
-					memory_limit), 2));
+	if (memory_usage <= memory_limit)
+		return;
+
+	// Look for the last filter if it is LZMA2 or LZMA1, so we can make
+	// it use less RAM. With other filters we don't know what to do.
+	size_t i = 0;
+	while (filters[i].id != LZMA_FILTER_LZMA2
+			&& filters[i].id != LZMA_FILTER_LZMA1) {
+		if (filters[i].id == LZMA_VLI_UNKNOWN)
+			memlimit_too_small(memory_usage);
+
+		++i;
 	}
 
-/*
-	// Limit the number of worker threads so that memory usage
-	// limit isn't exceeded.
-	assert(memory_usage > 0);
-	size_t thread_limit = memory_limit / memory_usage;
-	if (thread_limit == 0)
-		thread_limit = 1;
+	// Decrease the dictionary size until we meet the memory
+	// usage limit. First round down to full mebibytes.
+	lzma_options_lzma *opt = filters[i].options;
+	const uint32_t orig_dict_size = opt->dict_size;
+	opt->dict_size &= ~((UINT32_C(1) << 20) - 1);
+	while (true) {
+		// If it is below 1 MiB, auto-adjusting failed. We could be
+		// more sophisticated and scale it down even more, but let's
+		// see if many complain about this version.
+		//
+		// FIXME: Displays the scaled memory usage instead
+		// of the original.
+		if (opt->dict_size < (UINT32_C(1) << 20))
+			memlimit_too_small(memory_usage);
 
-	if (opt_threads > thread_limit)
-		opt_threads = thread_limit;
-*/
+		memory_usage = lzma_raw_encoder_memusage(filters);
+		if (memory_usage == UINT64_MAX)
+			message_bug();
 
-	if (check_default) {
-		// The default check type is CRC64, but fallback to CRC32
-		// if CRC64 isn't supported by the copy of liblzma we are
-		// using. CRC32 is always supported.
-		check = LZMA_CHECK_CRC64;
-		if (!lzma_check_is_supported(check))
-			check = LZMA_CHECK_CRC32;
+		// Accept it if it is low enough.
+		if (memory_usage <= memory_limit)
+			break;
+
+		// Otherwise 1 MiB down and try again. I hope this
+		// isn't too slow method for cases where the original
+		// dict_size is very big.
+		opt->dict_size -= UINT32_C(1) << 20;
 	}
 
+	// Tell the user that we decreased the dictionary size.
+	message(V_WARNING, _("Adjusted LZMA%c dictionary size "
+			"from %s MiB to %s MiB to not exceed "
+			"the memory usage limit of %s MiB"),
+			filters[i].id == LZMA_FILTER_LZMA2
+				? '2' : '1',
+			uint64_to_str(orig_dict_size >> 20, 0),
+			uint64_to_str(opt->dict_size >> 20, 1),
+			uint64_to_str(round_up_to_mib(memory_limit), 2));
+
 	return;
 }
 
@@ -354,7 +393,14 @@ coder_init(file_pair *pair)
 			break;
 
 		case FORMAT_XZ:
-			ret = lzma_stream_encoder(&strm, filters, check);
+#ifdef HAVE_PTHREAD
+			if (hardware_threads_get() > 1)
+				ret = lzma_stream_encoder_mt(
+						&strm, &mt_options);
+			else
+#endif
+				ret = lzma_stream_encoder(
+						&strm, filters, check);
 			break;
 
 		case FORMAT_LZMA:
@@ -366,8 +412,9 @@ coder_init(file_pair *pair)
 			break;
 		}
 	} else {
-		const uint32_t flags = LZMA_TELL_UNSUPPORTED_CHECK
-				| LZMA_CONCATENATED;
+		uint32_t flags = LZMA_TELL_UNSUPPORTED_CHECK;
+		if (!opt_single_stream)
+			flags |= LZMA_CONCATENATED;
 
 		// We abuse FORMAT_AUTO to indicate unknown file format,
 		// for which we may consider passthru mode.
@@ -398,7 +445,7 @@ coder_init(file_pair *pair)
 
 		switch (init_format) {
 		case FORMAT_AUTO:
-			// Uknown file format. If --decompress --stdout
+			// Unknown file format. If --decompress --stdout
 			// --force have been given, then we copy the input
 			// as is to stdout. Checking for MODE_DECOMPRESS
 			// is needed, because we don't want to do use
@@ -459,8 +506,8 @@ coder_normal(file_pair *pair)
 	// Encoder needs to know when we have given all the input to it.
 	// The decoders need to know it too when we are using
 	// LZMA_CONCATENATED. We need to check for src_eof here, because
-	// the first input chunk has been already read, and that may
-	// have been the only chunk we will read.
+	// the first input chunk has been already read if decompressing,
+	// and that may have been the only chunk we will read.
 	lzma_action action = pair->src_eof ? LZMA_FINISH : LZMA_RUN;
 
 	lzma_ret ret;
@@ -468,6 +515,16 @@ coder_normal(file_pair *pair)
 	// Assume that something goes wrong.
 	bool success = false;
 
+	// block_remaining indicates how many input bytes to encode until
+	// finishing the current .xz Block. The Block size is set with
+	// --block-size=SIZE. It has an effect only when compressing
+	// to the .xz format. If block_remaining == UINT64_MAX, only
+	// a single block is created.
+	uint64_t block_remaining = UINT64_MAX;
+	if (hardware_threads_get() == 1 && opt_mode == MODE_COMPRESS
+			&& opt_format == FORMAT_XZ && opt_block_size > 0)
+		block_remaining = opt_block_size;
+
 	strm.next_out = out_buf.u8;
 	strm.avail_out = IO_BUFFER_SIZE;
 
@@ -476,14 +533,23 @@ coder_normal(file_pair *pair)
 		// end of file yet.
 		if (strm.avail_in == 0 && !pair->src_eof) {
 			strm.next_in = in_buf.u8;
-			strm.avail_in = io_read(
-					pair, &in_buf, IO_BUFFER_SIZE);
+			strm.avail_in = io_read(pair, &in_buf,
+					my_min(block_remaining,
+						IO_BUFFER_SIZE));
 
 			if (strm.avail_in == SIZE_MAX)
 				break;
 
-			if (pair->src_eof)
+			if (pair->src_eof) {
 				action = LZMA_FINISH;
+
+			} else if (block_remaining != UINT64_MAX) {
+				// Start a new Block after every
+				// opt_block_size bytes of input.
+				block_remaining -= strm.avail_in;
+				if (block_remaining == 0)
+					action = LZMA_FULL_FLUSH;
+			}
 		}
 
 		// Let liblzma do the actual work.
@@ -499,7 +565,12 @@ coder_normal(file_pair *pair)
 			strm.avail_out = IO_BUFFER_SIZE;
 		}
 
-		if (ret != LZMA_OK) {
+		if (ret == LZMA_STREAM_END && action == LZMA_FULL_FLUSH) {
+			// Start a new Block.
+			action = LZMA_RUN;
+			block_remaining = opt_block_size;
+
+		} else if (ret != LZMA_OK) {
 			// Determine if the return value indicates that we
 			// won't continue coding.
 			const bool stop = ret != LZMA_NO_CHECK
@@ -518,6 +589,11 @@ coder_normal(file_pair *pair)
 			}
 
 			if (ret == LZMA_STREAM_END) {
+				if (opt_single_stream) {
+					success = true;
+					break;
+				}
+
 				// Check that there is no trailing garbage.
 				// This is needed for LZMA_Alone and raw
 				// streams.
@@ -620,10 +696,15 @@ coder_run(const char *filename)
 	// Assume that something goes wrong.
 	bool success = false;
 
-	// Read the first chunk of input data. This is needed to detect
-	// the input file type (for now, only for decompression).
-	strm.next_in = in_buf.u8;
-	strm.avail_in = io_read(pair, &in_buf, IO_BUFFER_SIZE);
+	if (opt_mode == MODE_COMPRESS) {
+		strm.next_in = NULL;
+		strm.avail_in = 0;
+	} else {
+		// Read the first chunk of input data. This is needed
+		// to detect the input file type.
+		strm.next_in = in_buf.u8;
+		strm.avail_in = io_read(pair, &in_buf, IO_BUFFER_SIZE);
+	}
 
 	if (strm.avail_in != SIZE_MAX) {
 		// Initialize the coder. This will detect the file format
@@ -661,3 +742,13 @@ coder_run(const char *filename)
 
 	return;
 }
+
+
+#ifndef NDEBUG
+extern void
+coder_free(void)
+{
+	lzma_end(&strm);
+	return;
+}
+#endif

src/xz/coder.h

@@ -41,12 +41,18 @@ extern enum format_type opt_format;
 /// they exceed the memory usage limit.
 extern bool opt_auto_adjust;
 
+/// If true, stop after decoding the first stream.
+extern bool opt_single_stream;
+
+/// If non-zero, start a new .xz Block after every opt_block_size bytes
+/// of input. This has an effect only when compressing to the .xz format.
+extern uint64_t opt_block_size;
 
 /// Set the integrity check type used when compressing
 extern void coder_set_check(lzma_check check);
 
 /// Set preset number
-extern void coder_set_preset(size_t new_preset);
+extern void coder_set_preset(uint32_t new_preset);
 
 /// Enable extreme mode
 extern void coder_set_extreme(void);
@@ -59,3 +65,8 @@ extern void coder_set_compression_settings(void);
 
 /// Compress or decompress the given file
 extern void coder_run(const char *filename);
+
+#ifndef NDEBUG
+/// Free the memory allocated for the coder and kill the worker threads.
+extern void coder_free(void);
+#endif

src/xz/file_io.c

@@ -53,7 +53,7 @@ static bool io_write_buf(file_pair *pair, const uint8_t *buf, size_t size);
 extern void
 io_init(void)
 {
-	// Make sure that stdin, stdout, and and stderr are connected to
+	// Make sure that stdin, stdout, and stderr are connected to
 	// a valid file descriptor. Exit immediately with exit code ERROR
 	// if we cannot make the file descriptors valid. Maybe we should
 	// print an error message, but our stderr could be screwed anyway.
@@ -68,8 +68,7 @@ io_init(void)
 #ifdef __DJGPP__
 	// Avoid doing useless things when statting files.
 	// This isn't important but doesn't hurt.
-	_djstat_flags = _STAT_INODE | _STAT_EXEC_EXT
-			| _STAT_EXEC_MAGIC | _STAT_DIRSIZE;
+	_djstat_flags = _STAT_EXEC_EXT | _STAT_EXEC_MAGIC | _STAT_DIRSIZE;
 #endif
 
 	return;
@@ -293,6 +292,10 @@ io_open_src_real(file_pair *pair)
 		pair->src_fd = STDIN_FILENO;
 #ifdef TUKLIB_DOSLIKE
 		setmode(STDIN_FILENO, O_BINARY);
+#endif
+#ifdef HAVE_POSIX_FADVISE
+		// It will fail if stdin is a pipe and that's fine.
+		(void)posix_fadvise(STDIN_FILENO, 0, 0, POSIX_FADV_SEQUENTIAL);
 #endif
 		return false;
 	}
@@ -448,8 +451,18 @@ io_open_src_real(file_pair *pair)
 
 	// Stat the source file. We need the result also when we copy
 	// the permissions, and when unlinking.
+	//
+	// NOTE: Use stat() instead of fstat() with DJGPP, because
+	// then we have a better chance to get st_ino value that can
+	// be used in io_open_dest_real() to prevent overwriting the
+	// source file.
+#ifdef __DJGPP__
+	if (stat(pair->src_name, &pair->src_st))
+		goto error_msg;
+#else
 	if (fstat(pair->src_fd, &pair->src_st))
 		goto error_msg;
+#endif
 
 	if (S_ISDIR(pair->src_st.st_mode)) {
 		message_warning(_("%s: Is a directory, skipping"),
@@ -497,6 +510,17 @@ io_open_src_real(file_pair *pair)
 	}
 #endif
 
+#ifdef HAVE_POSIX_FADVISE
+	const int fadvise_ret = posix_fadvise(
+			pair->src_fd, 0, 0, POSIX_FADV_SEQUENTIAL);
+
+	// It shouldn't fail, but if it does anyway, it doesn't matter.
+	// Check it with an assertion so that if something gets messed
+	// up in the future, it will get caught when debugging is enabled.
+	assert(fadvise_ret == 0);
+	(void)fadvise_ret;
+#endif
+
 	return false;
 
 error_msg:
@@ -584,6 +608,28 @@ io_open_dest_real(file_pair *pair)
 		if (pair->dest_name == NULL)
 			return true;
 
+#ifdef __DJGPP__
+		struct stat st;
+		if (stat(pair->dest_name, &st) == 0) {
+			// Check that it isn't a special file like "prn".
+			if (st.st_dev == -1) {
+				message_error("%s: Refusing to write to "
+						"a DOS special file",
+						pair->dest_name);
+				return true;
+			}
+
+			// Check that we aren't overwriting the source file.
+			if (st.st_dev == pair->src_st.st_dev
+					&& st.st_ino == pair->src_st.st_ino) {
+				message_error("%s: Output file is the same "
+						"as the input file",
+						pair->dest_name);
+				return true;
+			}
+		}
+#endif
+
 		// If --force was used, unlink the target file first.
 		if (opt_force && unlink(pair->dest_name) && errno != ENOENT) {
 			message_error(_("%s: Cannot remove: %s"),
@@ -606,17 +652,19 @@ io_open_dest_real(file_pair *pair)
 		}
 	}
 
-	// If this really fails... well, we have a safe fallback.
+#ifndef TUKLIB_DOSLIKE
+	// dest_st isn't used on DOS-like systems except as a dummy
+	// argument to io_unlink(), so don't fstat() on such systems.
 	if (fstat(pair->dest_fd, &pair->dest_st)) {
-#if defined(__VMS)
+		// If fstat() really fails, we have a safe fallback here.
+#	if defined(__VMS)
 		pair->dest_st.st_ino[0] = 0;
 		pair->dest_st.st_ino[1] = 0;
 		pair->dest_st.st_ino[2] = 0;
-#elif !defined(TUKLIB_DOSLIKE)
+#	else
 		pair->dest_st.st_dev = 0;
 		pair->dest_st.st_ino = 0;
-#endif
-#ifndef TUKLIB_DOSLIKE
+#	endif
 	} else if (try_sparse && opt_mode == MODE_DECOMPRESS) {
 		// When writing to standard output, we need to be extra
 		// careful:
@@ -674,8 +722,8 @@ io_open_dest_real(file_pair *pair)
 		}
 
 		pair->dest_try_sparse = true;
-#endif
 	}
+#endif
 
 	return false;
 }

src/xz/hardware.c

@@ -14,9 +14,9 @@
 #include "tuklib_cpucores.h"
 
 
-/// Maximum number of free *coder* threads. This can be set with
+/// Maximum number of worker threads. This can be set with
 /// the --threads=NUM command line option.
-static uint32_t threadlimit;
+static uint32_t threads_max = 1;
 
 /// Memory usage limit for compression
 static uint64_t memlimit_compress;
@@ -29,15 +29,16 @@ static uint64_t total_ram;
 
 
 extern void
-hardware_threadlimit_set(uint32_t new_threadlimit)
+hardware_threads_set(uint32_t n)
 {
-	if (new_threadlimit == 0) {
-		// The default is the number of available CPU cores.
-		threadlimit = tuklib_cpucores();
-		if (threadlimit == 0)
-			threadlimit = 1;
+	if (n == 0) {
+		// Automatic number of threads was requested.
+		// Use the number of available CPU cores.
+		threads_max = tuklib_cpucores();
+		if (threads_max == 0)
+			threads_max = 1;
 	} else {
-		threadlimit = new_threadlimit;
+		threads_max = n;
 	}
 
 	return;
@@ -45,9 +46,9 @@ hardware_threadlimit_set(uint32_t new_threadlimit)
 
 
 extern uint32_t
-hardware_threadlimit_get(void)
+hardware_threads_get(void)
 {
-	return threadlimit;
+	return threads_max;
 }
 
 
@@ -139,6 +140,5 @@ hardware_init(void)
 
 	// Set the defaults.
 	hardware_memlimit_set(0, true, true, false);
-	hardware_threadlimit_set(0);
 	return;
 }

src/xz/hardware.h

@@ -15,12 +15,11 @@
 extern void hardware_init(void);
 
 
-/// Set custom value for maximum number of coder threads.
-extern void hardware_threadlimit_set(uint32_t threadlimit);
+/// Set the maximum number of worker threads.
+extern void hardware_threads_set(uint32_t threadlimit);
 
-/// Get the maximum number of coder threads. Some additional helper threads
-/// are allowed on top of this).
-extern uint32_t hardware_threadlimit_get(void);
+/// Get the maximum number of worker threads.
+extern uint32_t hardware_threads_get(void);
 
 
 /// Set the memory usage limit. There are separate limits for compression

src/xz/main.c

@@ -275,6 +275,10 @@ main(int argc, char **argv)
 		list_totals();
 	}
 
+#ifndef NDEBUG
+	coder_free();
+#endif
+
 	// If we have got a signal, raise it to kill the program instead
 	// of calling tuklib_exit().
 	signals_exit();

src/xz/message.c

@@ -1108,7 +1108,10 @@ message_help(bool long_help)
 "  -f, --force         force overwrite of output file and (de)compress links\n"
 "  -c, --stdout        write to standard output and don't delete input files"));
 
-	if (long_help)
+	if (long_help) {
+		puts(_(
+"      --single-stream decompress only the first stream, and silently\n"
+"                      ignore possible remaining input data"));
 		puts(_(
 "      --no-sparse     do not create sparse files when decompressing\n"
 "  -S, --suffix=.SUF   use the suffix `.SUF' on compressed files\n"
@@ -1116,6 +1119,7 @@ message_help(bool long_help)
 "                      omitted, filenames are read from the standard input;\n"
 "                      filenames must be terminated with the newline character\n"
 "      --files0[=FILE] like --files but use the null character as terminator"));
+	}
 
 	if (long_help) {
 		puts(_("\n Basic file format and compression options:\n"));
@@ -1135,6 +1139,10 @@ message_help(bool long_help)
 "                      does not affect decompressor memory requirements"));
 
 	if (long_help) {
+		puts(_(
+"      --block-size=SIZE\n"
+"                      when compressing to the .xz format, start a new block\n"
+"                      after every SIZE bytes of input; 0=disabled (default)"));
 		puts(_( // xgettext:no-c-format
 "      --memlimit-compress=LIMIT\n"
 "      --memlimit-decompress=LIMIT\n"

src/xz/private.h

@@ -12,6 +12,8 @@
 
 #include "sysdefs.h"
 #include "mythread.h"
+
+#define LZMA_UNSTABLE
 #include "lzma.h"
 
 #include <sys/types.h>

src/xz/suffix.c

@@ -12,6 +12,10 @@
 
 #include "private.h"
 
+#ifdef __DJGPP__
+#	include <fcntl.h>
+#endif
+
 // For case-insensitive filename suffix on case-insensitive systems
 #if defined(TUKLIB_DOSLIKE) || defined(__VMS)
 #	define strcmp strcasecmp
@@ -21,12 +25,6 @@
 static char *custom_suffix = NULL;
 
 
-struct suffix_pair {
-	const char *compressed;
-	const char *uncompressed;
-};
-
-
 /// \brief      Test if the char is a directory separator
 static bool
 is_dir_sep(char c)
@@ -51,6 +49,31 @@ has_dir_sep(const char *str)
 }
 
 
+#ifdef __DJGPP__
+/// \brief      Test for special suffix used for 8.3 short filenames (SFN)
+///
+/// \return     If str matches *.?- or *.??-, true is returned. Otherwise
+///             false is returned.
+static bool
+has_sfn_suffix(const char *str, size_t len)
+{
+	if (len >= 4 && str[len - 1] == '-' && str[len - 2] != '.'
+			&& !is_dir_sep(str[len - 2])) {
+		// *.?-
+		if (str[len - 3] == '.')
+			return !is_dir_sep(str[len - 4]);
+
+		// *.??-
+		if (len >= 5 && !is_dir_sep(str[len - 3])
+				&& str[len - 4] == '.')
+			return !is_dir_sep(str[len - 5]);
+	}
+
+	return false;
+}
+#endif
+
+
 /// \brief      Checks if src_name has given compressed_suffix
 ///
 /// \param      suffix      Filename suffix to look for
@@ -86,10 +109,16 @@ test_suffix(const char *suffix, const char *src_name, size_t src_len)
 static char *
 uncompressed_name(const char *src_name, const size_t src_len)
 {
-	static const struct suffix_pair suffixes[] = {
+	static const struct {
+		const char *compressed;
+		const char *uncompressed;
+	} suffixes[] = {
 		{ ".xz",    "" },
 		{ ".txz",   ".tar" }, // .txz abbreviation for .txt.gz is rare.
 		{ ".lzma",  "" },
+#ifdef __DJGPP__
+		{ ".lzm",   "" },
+#endif
 		{ ".tlz",   ".tar" },
 		// { ".gz",    "" },
 		// { ".tgz",   ".tar" },
@@ -115,6 +144,17 @@ uncompressed_name(const char *src_name, const size_t src_len)
 				break;
 			}
 		}
+
+#ifdef __DJGPP__
+		// Support also *.?- -> *.? and *.??- -> *.?? on DOS.
+		// This is done also when long filenames are available
+		// to keep it easy to decompress files created when
+		// long filename support wasn't available.
+		if (new_len == 0 && has_sfn_suffix(src_name, src_len)) {
+			new_suffix = "";
+			new_len = src_len - 1;
+		}
+#endif
 	}
 
 	if (new_len == 0 && custom_suffix != NULL)
@@ -137,33 +177,47 @@ uncompressed_name(const char *src_name, const size_t src_len)
 }
 
 
+/// This message is needed in multiple places in compressed_name(),
+/// so the message has been put into its own function.
+static void
+msg_suffix(const char *src_name, const char *suffix)
+{
+	message_warning(_("%s: File already has `%s' suffix, skipping"),
+			src_name, suffix);
+	return;
+}
+
+
 /// \brief      Appends suffix to src_name
 ///
 /// In contrast to uncompressed_name(), we check only suffixes that are valid
 /// for the specified file format.
 static char *
-compressed_name(const char *src_name, const size_t src_len)
+compressed_name(const char *src_name, size_t src_len)
 {
 	// The order of these must match the order in args.h.
-	static const struct suffix_pair all_suffixes[][3] = {
+	static const char *const all_suffixes[][4] = {
 		{
-			{ ".xz",    "" },
-			{ ".txz",   ".tar" },
-			{ NULL, NULL }
+			".xz",
+			".txz",
+			NULL
 		}, {
-			{ ".lzma",  "" },
-			{ ".tlz",   ".tar" },
-			{ NULL,     NULL }
+			".lzma",
+#ifdef __DJGPP__
+			".lzm",
+#endif
+			".tlz",
+			NULL
 /*
 		}, {
-			{ ".gz",    "" },
-			{ ".tgz",   ".tar" },
-			{ NULL,     NULL }
+			".gz",
+			".tgz",
+			NULL
 */
 		}, {
 			// --format=raw requires specifying the suffix
 			// manually or using stdout.
-			{ NULL,     NULL }
+			NULL
 		}
 	};
 
@@ -171,14 +225,29 @@ compressed_name(const char *src_name, const size_t src_len)
 	assert(opt_format != FORMAT_AUTO);
 
 	const size_t format = opt_format - 1;
-	const struct suffix_pair *const suffixes = all_suffixes[format];
+	const char *const *suffixes = all_suffixes[format];
 
-	for (size_t i = 0; suffixes[i].compressed != NULL; ++i) {
-		if (test_suffix(suffixes[i].compressed, src_name, src_len)
-				!= 0) {
-			message_warning(_("%s: File already has `%s' "
-					"suffix, skipping"), src_name,
-					suffixes[i].compressed);
+	// Look for known filename suffixes and refuse to compress them.
+	for (size_t i = 0; suffixes[i] != NULL; ++i) {
+		if (test_suffix(suffixes[i], src_name, src_len) != 0) {
+			msg_suffix(src_name, suffixes[i]);
+			return NULL;
+		}
+	}
+
+#ifdef __DJGPP__
+	// Recognize also the special suffix that is used when long
+	// filename (LFN) support isn't available. This suffix is
+	// recognized on LFN systems too.
+	if (opt_format == FORMAT_XZ && has_sfn_suffix(src_name, src_len)) {
+		msg_suffix(src_name, "-");
+		return NULL;
+	}
+#endif
+
+	if (custom_suffix != NULL) {
+		if (test_suffix(custom_suffix, src_name, src_len) != 0) {
+			msg_suffix(src_name, custom_suffix);
 			return NULL;
 		}
 	}
@@ -193,8 +262,102 @@ compressed_name(const char *src_name, const size_t src_len)
 	}
 
 	const char *suffix = custom_suffix != NULL
-			? custom_suffix : suffixes[0].compressed;
-	const size_t suffix_len = strlen(suffix);
+			? custom_suffix : suffixes[0];
+	size_t suffix_len = strlen(suffix);
+
+#ifdef __DJGPP__
+	if (!_use_lfn(src_name)) {
+		// Long filename (LFN) support isn't available and we are
+		// limited to 8.3 short filenames (SFN).
+		//
+		// Look for suffix separator from the filename, and make sure
+		// that it is in the filename, not in a directory name.
+		const char *sufsep = strrchr(src_name, '.');
+		if (sufsep == NULL || sufsep[1] == '\0'
+				|| has_dir_sep(sufsep)) {
+			// src_name has no filename extension.
+			//
+			// Examples:
+			// xz foo         -> foo.xz
+			// xz -F lzma foo -> foo.lzm
+			// xz -S x foo    -> foox
+			// xz -S x foo.   -> foo.x
+			// xz -S x.y foo  -> foox.y
+			// xz -S .x foo   -> foo.x
+			// xz -S .x foo.  -> foo.x
+			//
+			// Avoid double dots:
+			if (sufsep != NULL && sufsep[1] == '\0'
+					&& suffix[0] == '.')
+				--src_len;
+
+		} else if (custom_suffix == NULL
+				&& strcasecmp(sufsep, ".tar") == 0) {
+			// ".tar" is handled specially.
+			//
+			// Examples:
+			// xz foo.tar          -> foo.txz
+			// xz -F lzma foo.tar  -> foo.tlz
+			static const char *const tar_suffixes[] = {
+				".txz",
+				".tlz",
+				// ".tgz",
+			};
+			suffix = tar_suffixes[format];
+			suffix_len = 4;
+			src_len -= 4;
+
+		} else {
+			if (custom_suffix == NULL && opt_format == FORMAT_XZ) {
+				// Instead of the .xz suffix, use a single
+				// character at the end of the filename
+				// extension. This is to minimize name
+				// conflicts when compressing multiple files
+				// with the same basename. E.g. foo.txt and
+				// foo.exe become foo.tx- and foo.ex-. Dash
+				// is rare as the last character of the
+				// filename extension, so it seems to be
+				// quite safe choice and it stands out better
+				// in directory listings than e.g. x. For
+				// comparison, gzip uses z.
+				suffix = "-";
+				suffix_len = 1;
+			}
+
+			if (suffix[0] == '.') {
+				// The first character of the suffix is a dot.
+				// Throw away the original filename extension
+				// and replace it with the new suffix.
+				//
+				// Examples:
+				// xz -F lzma foo.txt  -> foo.lzm
+				// xz -S .x  foo.txt   -> foo.x
+				src_len = sufsep - src_name;
+
+			} else {
+				// The first character of the suffix is not
+				// a dot. Preserve the first 0-2 characters
+				// of the original filename extension.
+				//
+				// Examples:
+				// xz foo.txt         -> foo.tx-
+				// xz -S x  foo.c     -> foo.cx
+				// xz -S ab foo.c     -> foo.cab
+				// xz -S ab foo.txt   -> foo.tab
+				// xz -S abc foo.txt  -> foo.abc
+				//
+				// Truncate the suffix to three chars:
+				if (suffix_len > 3)
+					suffix_len = 3;
+
+				// If needed, overwrite 1-3 characters.
+				if (strlen(sufsep) > 4 - suffix_len)
+					src_len = sufsep - src_name
+							+ 4 - suffix_len;
+			}
+		}
+	}
+#endif
 
 	char *dest_name = xmalloc(src_len + suffix_len + 1);
 

src/xz/xz.1

@@ -5,7 +5,7 @@
 .\" This file has been put into the public domain.
 .\" You can do whatever you want with this file.
 .\"
-.TH XZ 1 "2010-10-04" "Tukaani" "XZ Utils"
+.TH XZ 1 "2011-04-12" "Tukaani" "XZ Utils"
 .
 .SH NAME
 xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files
@@ -434,6 +434,29 @@ standard output instead of a file.
 This implies
 .BR \-\-keep .
 .TP
+.B \-\-single\-stream
+Decompress only the first
+.B .xz
+stream, and
+silently ignore possible remaining input data following the stream.
+Normally such trailing garbage makes
+.B xz
+display an error.
+.IP ""
+.B xz
+never decompresses more than one stream from
+.B .lzma
+files or raw streams, but this option still makes
+.B xz
+ignore the possible trailing data after the
+.B .lzma
+file or raw stream.
+.IP ""
+This option has no effect if the operation mode is not
+.B \-\-decompress
+or
+.BR \-\-test .
+.TP
 .B \-\-no\-sparse
 Disable creation of sparse files.
 By default, if decompressing into a regular file,
@@ -777,6 +800,15 @@ These are provided only for backwards compatibility
 with LZMA Utils.
 Avoid using these options.
 .TP
+.BI \-\-block\-size= size
+When compressing to the
+.B .xz
+format, split the input data into blocks of
+.I size
+bytes.
+The blocks are compressed independently from each other.
+.\" FIXME: Explain how to his can be used for random access and threading.
+.TP
 .BI \-\-memlimit\-compress= limit
 Set a memory usage limit for compression.
 If this option is specified multiple times,
@@ -866,7 +898,7 @@ This is equivalent to specifying \fB\-\-memlimit\-compress=\fIlimit
 \fB\-\-memlimit\-decompress=\fIlimit\fR.
 .TP
 .B \-\-no\-adjust
-Display an error and exit if the compression settings exceed the
+Display an error and exit if the compression settings exceed
 the memory usage limit.
 The default is to adjust the settings downwards so
 that the memory usage limit is not exceeded.
@@ -875,24 +907,30 @@ Automatic adjusting is always disabled when creating raw streams
 .TP
 \fB\-T\fR \fIthreads\fR, \fB\-\-threads=\fIthreads
 Specify the number of worker threads to use.
+Setting
+.I threads
+to a special value
+.B 0
+makes
+.B xz
+use as many threads as there are CPU cores on the system.
 The actual number of threads can be less than
 .I threads
+if the input file is not big enough
+for threading with the given settings or
 if using more threads would exceed the memory usage limit.
 .IP ""
-.B "Multithreaded compression and decompression are not"
-.B "implemented yet, so this option has no effect for now."
+Currently the only threading method is to split the input into
+blocks and compress them independently from each other.
+The default block size depends on the compression level and
+can be overriden with the
+.BI \-\-block\-size= size
+option.
 .IP ""
-.B "As of writing (2010-09-27), it hasn't been decided"
-.B "if threads will be used by default on multicore systems"
-.B "once support for threading has been implemented."
-.B "Comments are welcome."
-The complicating factor is that using many threads
-will increase the memory usage dramatically.
-Note that if multithreading will be the default,
-it will probably be done so that single-threaded and
-multithreaded modes produce the same output,
-so compression ratio won't be significantly affected
-if threading will be enabled by default.
+.B "It is possible that the details of this option change before"
+.B "the next stable XZ Utils release."
+.B "This may include the meaning of the special value 0."
+.\" FIXME
 .
 .SS "Custom compressor filter chains"
 A custom filter chain allows specifying
@@ -2142,7 +2180,9 @@ If there is data left after the first
 .B .lzma
 stream,
 .B xz
-considers the file to be corrupt.
+considers the file to be corrupt unless
+.B \-\-single\-stream
+was used.
 This may break obscure scripts which have
 assumed that trailing garbage is ignored.
 .

tests/files/README

@@ -87,6 +87,10 @@
     uncompressed with dictionary reset, and third is LZMA with new
     properties but without dictionary reset.
 
+    good-1-lzma2-5.xz has an empty LZMA2 stream with only the end of
+    payload marker. XZ Utils 5.0.1 and older incorrectly see this file
+    as corrupt.
+
     good-1-3delta-lzma2.xz has three Delta filters and LZMA2.
 
 

windows/INSTALL-Windows.txt

@@ -133,7 +133,7 @@ Building XZ Utils
 Using a snapshot from the Git repository
 
     To use a snapshot, the build system files need to be generated with
-    autogen.sh or "autoreconf -fi" before trying to build using the the
+    autogen.sh or "autoreconf -fi" before trying to build using the
     above build instructions. You can install the relevant extra packages
     from MinGW or use Cygwin or use e.g. a GNU/Linux system to create a
     source package with the required build system files.