Disable Subblock filter from test_compress.sh since it is

disabled by default in configure.ac.

AUTHORS

@@ -1,6 +1,6 @@
 
-Authors of LZMA Utils
----------------------
+Authors of XZ Utils
+===================
 
 Igor Pavlov
   * designed LZMA as an algorithm;

COPYING

@@ -5,7 +5,8 @@ LZMA Utils Licenses
     Different licenses apply to different files in this package. Here
     is a rough summary of which license apply to which parts of this
     package (but check the individual files to be sure!):
-      - Everything under src/liblzma/check is public domain.
+      - Everything under src/liblzma/check and tests/files is public
+        domain.
       - Everything else under the src directory is under the GNU LGPL
         2.1 or (at your opinion) any later version.
       - Outside the src directory, there are some files that are under

Makefile.am

@@ -12,6 +12,7 @@
 ##  Lesser General Public License for more details.
 ##
 
+DIST_SUBDIRS = lib src po tests debug
 SUBDIRS =
 
 if COND_GNULIB
@@ -20,9 +21,7 @@ endif
 
 SUBDIRS += src po tests
 
-
 EXTRA_DIST = \
-	m4 \
 	config.rpath  \
 	Doxyfile.in \
 	extra \
@@ -32,4 +31,8 @@ EXTRA_DIST = \
 	COPYING.LGPLv2.1
 
 ACLOCAL_AMFLAGS = -I m4
-AUTOMAKE_OPTIONS = foreign
+
+# This works with GNU tar and gives cleaner package than normal 'make dist'.
+mydist:
+	TAR_OPTIONS='--owner=0 --group=0 --numeric-owner --mode=u+rw,go+r-w' \
+		$(MAKE) dist-gzip

README

@@ -1,13 +1,13 @@
 
-LZMA Utils
-----------
+XZ Utils
+--------
 
 Warning
 
-    This is an early alpha version. Don't trust the files produced by
+    This is a beta version. Don't trust the files produced by
     this version of the software - not even if the software can
     uncompress the files properly! This is because the file format
-    isn't completely frozen yet.
+    isn't officially frozen yet although changes are very unlikely.
 
     So please test a lot, but don't use for anything serious yet.
 
@@ -18,19 +18,19 @@ Overview
     Igor Pavlov as part of 7-Zip. It provides high compression ratio
     while keeping the decompression speed fast.
 
-    LZMA Utils are an attempt to make LZMA compression easy to use
+    XZ Utils are an attempt to make LZMA compression easy to use
     on free (as in freedom) operating systems. This is achieved by
     providing tools and libraries which are similar to use than the
     equivalents of the most popular existing compression algorithms.
 
-    LZMA Utils consist of a few relatively separate parts:
+    XZ Utils consist of a few relatively separate parts:
       * liblzma is an encoder/decoder library with support for several
         filters (algorithm implementations). The primary filter is LZMA.
-      * libzfile enables reading from and writing to gzip, bzip2 and
-        LZMA compressed and uncompressed files with an API similar to
-        the standard ANSI-C file I/O.
+      * libzfile (or whatever the name will be) enables reading from and
+        writing to gzip, bzip2 and LZMA compressed and uncompressed files
+        with an API similar to the standard ANSI-C file I/O.
         [ NOTE: libzfile is not implemented yet. ]
-      * lzma command line tool has almost identical syntax than gzip
+      * xz command line tool has almost identical syntax than gzip
         and bzip2. It makes LZMA easy for average users, but also
         provides advanced options to finetune the compression settings.
       * A few shell scripts make diffing and grepping LZMA compressed
@@ -39,15 +39,15 @@ Overview
 
 Supported platforms
 
-    LZMA Utils are developed on GNU+Linux, but they should work at
+    XZ Utils are developed on GNU+Linux, but they should work at
     least on *BSDs and Solaris. They probably work on some other
     POSIX-like operating systems too.
 
-    If you use GCC to compile LZMA Utils, you need at least version
+    If you use GCC to compile XZ Utils, you need at least version
     3.x.x. GCC version 2.xx.x doesn't support some C99 features used
-    in LZMA Utils source code, thus GCC 2 won't compile LZMA Utils.
+    in XZ Utils source code, thus GCC 2 won't compile XZ Utils.
 
-    If you have written patches to make LZMA Utils to work on previously
+    If you have written patches to make XZ Utils to work on previously
     unsupported platform, please send the patches to me! I will consider
     including them to the official version. It's nice to minimize the
     need of third-party patching.
@@ -59,6 +59,35 @@ Supported platforms
     in C89 or C++.
 
 
+Version numbering
+
+    The version number of XZ Utils has absolutely nothing to do with
+    the version number of LZMA SDK or 7-Zip. The new version number
+    format of XZ Utils is X.Y.ZS:
+
+      - X is the major version. When this is incremented, the library
+        API and ABI break.
+
+      - Y is the minor version. It is incremented when new features are
+        added without breaking existing API or ABI. Even Y indicates
+        stable release and odd Y indicates unstable (alpha or beta
+        version).
+
+      - Z is the revision. This has different meaning for stable and
+        unstable releases:
+          * Stable: Z is incremented when bugs get fixed without adding
+            any new features.
+          * Unstable: Z is just a counter. API or ABI of features added
+            in earlier unstable releases having the same X.Y may break.
+
+      - S indicates stability of the release. It is missing from the
+        stable releases where Y is an even number. When Y is odd, S
+        is either "alpha" or "beta" to make it very clear that such
+        versions are not stable releases. The same X.Y.Z combination is
+        not used for more than one stability level i.e. after X.Y.Zalpha,
+        the next version can be X.Y.(Z+1)beta but not X.Y.Zbeta.
+
+
 configure options
 
     If you are not familiar with `configure' scripts, read the file
@@ -113,7 +142,11 @@ configure options
 
                 All the assembler code in liblzma is position-independent
                 code, which is suitable for use in shared libraries and
-                position-independent executables.
+                position-independent executables. So far only i386
+                instructions are used, but the code is optimized for i686
+                class CPUs. If you are compiling liblzma exclusively for
+                pre-i686 systems, you may want to disable the assembler
+                code.
 
     --enable-small
                 Omits precomputed tables. This makes liblzma a few KiB

THANKS

@@ -6,17 +6,25 @@ Some people have helped more, some less, some don't even know they have
 been helpful, but nevertheless everyone's help has been important. :-)
 In alphabetical order:
   - Mark Adler
+  - Nelson H. F. Beebe
   - Anders F. Björklund
   - İsmail Dönmez
   - Jean-loup Gailly
   - Per Øyvind Karlsen
   - Ville Koskinen
+  - Stephan Kulow
+  - Andraž 'ruskie' Levstik
   - Jim Meyering
   - Igor Pavlov
   - Mikko Pouru
+  - Bernhard Reutner-Fischer
   - Alexandre Sauvé
+  - Andreas Schwab
   - Julian Seward
+  - Paul Townsend
   - Mohammed Adnène Trojette
+  - Ralf Wildenhues
+  - Andreas Zieringer
 
 Also thanks to all the people who have participated the Tukaani project
 and others who I have forgot.

configure.ac

@@ -22,45 +22,50 @@
 # of malloc(), stat(), or lstat(), since we don't use those functions in
 # a way that would cause the problems the autoconf macros check.
 
-AC_PREREQ(2.61)
-
-# [LZMA] instead of [LZMA utils] since I prefer to have lzma-version.tar.gz
-# instead of lzma-utils-version.tar.gz.
-AC_INIT([LZMA], [4.42.2alpha], [lasse.collin@tukaani.org])
+AC_PREREQ([2.61])
 
+AC_INIT([XZ Utils], [4.999.7beta], [lasse.collin@tukaani.org], [xz])
 AC_CONFIG_SRCDIR([src/liblzma/common/common.h])
 AC_CONFIG_HEADER([config.h])
 
 echo
-echo "LZMA Utils $PACKAGE_VERSION"
+echo "$PACKAGE_STRING"
 
 echo
 echo "System type:"
 # This is needed to know if assembler optimizations can be used.
 AC_CANONICAL_HOST
 
+
 echo
 echo "Configure options:"
 
-# Enable/disable debugging code:
+
+#############
+# Debugging #
+#############
+
 AC_MSG_CHECKING([if debugging code should be compiled])
-AC_ARG_ENABLE(debug, AC_HELP_STRING([--enable-debug], [Enable debugging code.]),
+AC_ARG_ENABLE([debug], AC_HELP_STRING([--enable-debug], [Enable debugging code.]),
 	[], enable_debug=no)
 if test "x$enable_debug" = xyes; then
-	CFLAGS="-g $CFLAGS"
 	AC_MSG_RESULT([yes])
 else
-	AC_DEFINE(NDEBUG, 1, [Define to disable debugging code.])
+	AC_DEFINE([NDEBUG], [1], [Define to 1 to disable debugging code.])
 	AC_MSG_RESULT([no])
 fi
 
-# Enable/disable the encoder components:
+
+###########
+# Encoder #
+###########
+
 AC_MSG_CHECKING([if encoder components should be built])
-AC_ARG_ENABLE(encoder, AC_HELP_STRING([--disable-encoder],
+AC_ARG_ENABLE([encoder], AC_HELP_STRING([--disable-encoder],
 		[Do not build the encoder components.]),
 	[], enable_encoder=yes)
 if test "x$enable_encoder" = xyes; then
-	AC_DEFINE([HAVE_ENCODER], 1,
+	AC_DEFINE([HAVE_ENCODER], [1],
 		[Define to 1 if encoder components are enabled.])
 	AC_MSG_RESULT([yes])
 else
@@ -68,13 +73,17 @@ else
 fi
 AM_CONDITIONAL(COND_MAIN_ENCODER, test "x$enable_encoder" = xyes)
 
-# Enable/disable the decoder components:
+
+###########
+# Decoder #
+###########
+
 AC_MSG_CHECKING([if decoder components should be built])
-AC_ARG_ENABLE(decoder, AC_HELP_STRING([--disable-decoder],
+AC_ARG_ENABLE([decoder], AC_HELP_STRING([--disable-decoder],
 		[Do not build the decoder components.]),
 	[], enable_decoder=yes)
 if test "x$enable_decoder" = xyes; then
-	AC_DEFINE([HAVE_DECODER], 1,
+	AC_DEFINE([HAVE_DECODER], [1],
 		[Define to 1 if decoder components are enabled.])
 	AC_MSG_RESULT([yes])
 else
@@ -85,146 +94,146 @@ else
 fi
 AM_CONDITIONAL(COND_MAIN_DECODER, test "x$enable_decoder" = xyes)
 
-# Filters
-AC_MSG_CHECKING([which filters to build])
-AC_ARG_ENABLE(filters, AC_HELP_STRING([--enable-filters=],
-		[Comma-separated list of filters to build. Default=all.
-		Filters used in encoding are needed also in decoding.
-		Available filters: copy subblock x86 powerpc ia64
-		arm armthumb sparc delta lzma]),
-		[], [enable_filters=copy,subblock,x86,powerpc,ia64,arm,armthumb,sparc,delta,lzma])
-enable_filters=`echo "$enable_filters" | sed 's/,/ /g'`
-enable_filters_copy=no
-enable_filters_subblock=no
-enable_filters_x86=no
-enable_filters_powerpc=no
-enable_filters_ia64=no
-enable_filters_arm=no
-enable_filters_armthumb=no
-enable_filters_sparc=no
-enable_filters_delta=no
-enable_filters_lzma=no
-enable_simple_filters=no
-if test "x$enable_filters" = xno || test "x$enable_filters" = x; then
-	AC_MSG_RESULT([])
-	AC_MSG_ERROR([Please enable at least one filter.])
+
+###########
+# Filters #
+###########
+
+m4_define([SUPPORTED_FILTERS], [lzma1,lzma2,subblock,delta,x86,powerpc,ia64,arm,armthumb,sparc])dnl
+m4_define([SIMPLE_FILTERS], [x86,powerpc,ia64,arm,armthumb,sparc])
+m4_define([LZ_FILTERS], [lzma1,lzma2])
+
+m4_foreach([NAME], [SUPPORTED_FILTERS],
+[enable_filter_[]NAME=no
+enable_encoder_[]NAME=no
+enable_decoder_[]NAME=no
+])dnl
+
+AC_MSG_CHECKING([which encoders to build])
+AC_ARG_ENABLE([encoders], AC_HELP_STRING([--enable-encoders=LIST],
+		[Comma-separated list of encoders to build. Default=all.
+		Available encoders:]
+			m4_translit(m4_defn([SUPPORTED_FILTERS]), [,], [ ])),
+	[], [enable_encoders=SUPPORTED_FILTERS])
+enable_encoders=`echo "$enable_encoders" | sed 's/,subblock//; s/,/ /g'`
+if test "x$enable_encoders" = xno || test "x$enable_encoders" = x; then
+	AC_MSG_RESULT([(none)])
 else
-	for arg in $enable_filters
+	for arg in $enable_encoders
 	do
-		case $arg in
-			copy)
-				enable_filters_copy=yes
-				AC_DEFINE([HAVE_FILTER_COPY], 1,
-					[Define to 1 if support for the
-					Copy filter is enabled.])
-				;;
-			subblock)
-				enable_filters_subblock=yes
-				AC_DEFINE([HAVE_FILTER_SUBBLOCK], 1,
-					[Define to 1 if support for the
-					Subblock filter is enabled.])
-				;;
-			x86)
-				enable_filters_x86=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_X86], 1,
-					[Define to 1 if support for the
-					x86 (BCJ) filter is enabled.])
-				;;
-			powerpc)
-				enable_filters_powerpc=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_POWERPC], 1,
-					[Define to 1 if support for the
-					PowerPC filter is enabled.])
-				;;
-			ia64)
-				enable_filters_ia64=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_IA64], 1,
-					[Define to 1 if support for the
-					IA64 filter is enabled.])
-				;;
-			arm)
-				enable_filters_arm=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_ARM], 1,
-					[Define to 1 if support for the
-					ARM filter is enabled.])
-				;;
-			armthumb)
-				enable_filters_armthumb=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_ARMTHUMB], 1,
-					[Define to 1 if support for the
-					ARMThumb filter is enabled.])
-				;;
-			sparc)
-				enable_filters_sparc=yes
-				enable_simple_filters=yes
-				AC_DEFINE([HAVE_FILTER_SPARC], 1,
-					[Define to 1 if support for the
-					SPARC filter is enabled.])
-				;;
-			delta)
-				enable_filters_delta=yes
-				AC_DEFINE([HAVE_FILTER_DELTA], 1,
-					[Define to 1 if support for the
-					Delta filter is enabled.])
-				;;
-			lzma)
-				enable_filters_lzma=yes
-				AC_DEFINE([HAVE_FILTER_LZMA], 1,
-					[Define to 1 if support for the
-					LZMA filter is enabled.])
-				;;
+		case $arg in m4_foreach([NAME], [SUPPORTED_FILTERS], [
+			NAME)
+				enable_filter_[]NAME=yes
+				enable_encoder_[]NAME=yes
+				AC_DEFINE(HAVE_ENCODER_[]m4_toupper(NAME), [1],
+				[Define to 1 if] NAME [encoder is enabled.])
+				;;])
 			*)
 				AC_MSG_RESULT([])
 				AC_MSG_ERROR([unknown filter: $arg])
 				;;
 		esac
 	done
-	AC_MSG_RESULT([$enable_filters])
+	AC_MSG_RESULT([$enable_encoders])
 fi
-if test "x$enable_simple_filters" = xyes ; then
-	AC_DEFINE([HAVE_FILTER_SIMPLE], 1, [Define to 1 if support for any
-		of the so called simple filters is enabled.])
-fi
-AM_CONDITIONAL(COND_FILTER_COPY, test "x$enable_filters_copy" = xyes)
-AM_CONDITIONAL(COND_FILTER_SUBBLOCK, test "x$enable_filters_subblock" = xyes)
-AM_CONDITIONAL(COND_FILTER_X86, test "x$enable_filters_x86" = xyes)
-AM_CONDITIONAL(COND_FILTER_POWERPC, test "x$enable_filters_powerpc" = xyes)
-AM_CONDITIONAL(COND_FILTER_IA64, test "x$enable_filters_ia64" = xyes)
-AM_CONDITIONAL(COND_FILTER_ARM, test "x$enable_filters_arm" = xyes)
-AM_CONDITIONAL(COND_FILTER_ARMTHUMB, test "x$enable_filters_armthumb" = xyes)
-AM_CONDITIONAL(COND_FILTER_SPARC, test "x$enable_filters_sparc" = xyes)
-AM_CONDITIONAL(COND_FILTER_DELTA, test "x$enable_filters_delta" = xyes)
-AM_CONDITIONAL(COND_FILTER_LZMA, test "x$enable_filters_lzma" = xyes)
-AM_CONDITIONAL(COND_MAIN_SIMPLE, test "x$enable_simple_filters" = xyes)
 
-# Which match finders should be enabled:
+AC_MSG_CHECKING([which decoders to build])
+AC_ARG_ENABLE([decoders], AC_HELP_STRING([--enable-decoders=LIST],
+		[Comma-separated list of decoders to build. Default=all.
+		Available decoders are the same as available encoders.]),
+	[], [enable_decoders=SUPPORTED_FILTERS])
+enable_decoders=`echo "$enable_decoders" | sed 's/,subblock//; s/,/ /g'`
+if test "x$enable_decoders" = xno || test "x$enable_decoders" = x; then
+	AC_MSG_RESULT([(none)])
+else
+	for arg in $enable_decoders
+	do
+		case $arg in m4_foreach([NAME], [SUPPORTED_FILTERS], [
+			NAME)
+				enable_filter_[]NAME=yes
+				enable_decoder_[]NAME=yes
+				AC_DEFINE(HAVE_DECODER_[]m4_toupper(NAME), [1],
+				[Define to 1 if] NAME [decoder is enabled.])
+				;;])
+			*)
+				AC_MSG_RESULT([])
+				AC_MSG_ERROR([unknown filter: $arg])
+				;;
+		esac
+	done
+
+	# LZMA2 requires that LZMA1 is enabled.
+	test "x$enable_encoder_lzma2" = xyes && enable_encoder_lzma1=yes
+	test "x$enable_decoder_lzma2" = xyes && enable_decoder_lzma1=yes
+
+	AC_MSG_RESULT([$enable_decoders])
+fi
+
+if test "x$enable_encoder_lzma2$enable_encoder_lzma1" = xyesno \
+		|| test "x$enable_decoder_lzma2$enable_decoder_lzma1" = xyesno; then
+	AC_MSG_ERROR([LZMA2 requires that LZMA1 is also enabled.])
+fi
+
+m4_foreach([NAME], [SUPPORTED_FILTERS],
+[AM_CONDITIONAL(COND_FILTER_[]m4_toupper(NAME), test "x$enable_filter_[]NAME" = xyes)
+AM_CONDITIONAL(COND_ENCODER_[]m4_toupper(NAME), test "x$enable_encoder_[]NAME" = xyes)
+AM_CONDITIONAL(COND_DECODER_[]m4_toupper(NAME), test "x$enable_decoder_[]NAME" = xyes)
+])dnl
+
+# The so called "simple filters" share common code.
+enable_filter_simple=no
+enable_encoder_simple=no
+enable_decoder_simple=no
+m4_foreach([NAME], [SIMPLE_FILTERS],
+[test "x$enable_filter_[]NAME" = xyes && enable_filter_simple=yes
+test "x$enable_encoder_[]NAME" = xyes && enable_encoder_simple=yes
+test "x$enable_decoder_[]NAME" = xyes && enable_decoder_simple=yes
+])dnl
+AM_CONDITIONAL(COND_FILTER_SIMPLE, test "x$enable_filter_simple" = xyes)
+AM_CONDITIONAL(COND_ENCODER_SIMPLE, test "x$enable_encoder_simple" = xyes)
+AM_CONDITIONAL(COND_DECODER_SIMPLE, test "x$enable_decoder_simple" = xyes)
+
+# LZ-based filters share common code.
+enable_filter_lz=no
+enable_encoder_lz=no
+enable_decoder_lz=no
+m4_foreach([NAME], [LZ_FILTERS],
+[test "x$enable_filter_[]NAME" = xyes && enable_filter_lz=yes
+test "x$enable_encoder_[]NAME" = xyes && enable_encoder_lz=yes
+test "x$enable_decoder_[]NAME" = xyes && enable_decoder_lz=yes
+])dnl
+AM_CONDITIONAL(COND_FILTER_LZ, test "x$enable_filter_lz" = xyes)
+AM_CONDITIONAL(COND_ENCODER_LZ, test "x$enable_encoder_lz" = xyes)
+AM_CONDITIONAL(COND_DECODER_LZ, test "x$enable_decoder_lz" = xyes)
+
+
+#################
+# Match finders #
+#################
+
+m4_define([SUPPORTED_MATCH_FINDERS], [hc3,hc4,bt2,bt3,bt4])
+
+m4_foreach([NAME], [SUPPORTED_MATCH_FINDERS],
+[enable_match_finder_[]NAME=no
+])
+
 AC_MSG_CHECKING([which match finders to build])
-AC_ARG_ENABLE(match-finders, AC_HELP_STRING([--enable-match-finders=],
+AC_ARG_ENABLE([match-finders], AC_HELP_STRING([--enable-match-finders=LIST],
 		[Comma-separated list of match finders to build. Default=all.
 		At least one match finder is required for encoding with
-		the LZMA filter.
-		Available match finders: hc3 hc4 bt2 bt3 bt4]), [],
-	[enable_match_finders=hc3,hc4,bt2,bt3,bt4])
+		the LZMA1 and LZMA2 filters. Available match finders:]
+		m4_translit(m4_defn([SUPPORTED_MATCH_FINDERS]), [,], [ ])), [],
+	[enable_match_finders=SUPPORTED_MATCH_FINDERS])
 enable_match_finders=`echo "$enable_match_finders" | sed 's/,/ /g'`
-enable_match_finders_hc3=no
-enable_match_finders_hc4=no
-enable_match_finders_bt2=no
-enable_match_finders_bt3=no
-enable_match_finders_bt4=no
-if test "x$enable_encoder" = xyes && test "x$enable_filters_lzma" = xyes ; then
+if test "x$enable_encoder" = xyes && test "x$enable_encoder_lz" = xyes ; then
 	for arg in $enable_match_finders
 		do
-		case $arg in
-			hc3) enable_match_finders_hc3=yes ;;
-			hc4) enable_match_finders_hc4=yes ;;
-			bt2) enable_match_finders_bt2=yes ;;
-			bt3) enable_match_finders_bt3=yes ;;
-			bt4) enable_match_finders_bt4=yes ;;
+		case $arg in m4_foreach([NAME], [SUPPORTED_MATCH_FINDERS], [
+			NAME)
+				enable_match_finder_[]NAME=yes
+				AC_DEFINE(HAVE_MF_[]m4_toupper(NAME), [1],
+				[Define to 1 to enable] NAME [match finder.])
+				;;])
 			*)
 				AC_MSG_RESULT([])
 				AC_MSG_ERROR([unknown match finder: $arg])
@@ -233,48 +242,39 @@ if test "x$enable_encoder" = xyes && test "x$enable_filters_lzma" = xyes ; then
 	done
 	AC_MSG_RESULT([$enable_match_finders])
 else
-	AC_MSG_RESULT([(none because not building the LZMA encoder)])
+	AC_MSG_RESULT([(none because not building any LZ-based encoder)])
 fi
-AM_CONDITIONAL(COND_MF_HC3, test "x$enable_match_finders_hc3" = xyes)
-AM_CONDITIONAL(COND_MF_HC4, test "x$enable_match_finders_hc4" = xyes)
-AM_CONDITIONAL(COND_MF_BT2, test "x$enable_match_finders_bt2" = xyes)
-AM_CONDITIONAL(COND_MF_BT3, test "x$enable_match_finders_bt3" = xyes)
-AM_CONDITIONAL(COND_MF_BT4, test "x$enable_match_finders_bt4" = xyes)
 
-# Which integrity checks to build
+
+####################
+# Integrity checks #
+####################
+
+m4_define([SUPPORTED_CHECKS], [crc32,crc64,sha256])
+
+m4_foreach([NAME], [SUPPORTED_FILTERS],
+[enable_check_[]NAME=no
+])dnl
+
 AC_MSG_CHECKING([which integrity checks to build])
-AC_ARG_ENABLE(checks, AC_HELP_STRING([--enable-checks=],
+AC_ARG_ENABLE([checks], AC_HELP_STRING([--enable-checks=LIST],
 		[Comma-separated list of integrity checks to build.
-		Default=all. Available integrity checks: crc32 crc64 sha256]),
-		[], [enable_checks=crc32,crc64,sha256])
+		Default=all. Available integrity checks:]
+		m4_translit(m4_defn([SUPPORTED_CHECKS]), [,], [ ])),
+	[], [enable_checks=SUPPORTED_CHECKS])
 enable_checks=`echo "$enable_checks" | sed 's/,/ /g'`
-enable_checks_crc32=no
-enable_checks_crc64=no
-enable_checks_sha256=no
 if test "x$enable_checks" = xno || test "x$enable_checks" = x; then
 	AC_MSG_RESULT([(none)])
 else
 	for arg in $enable_checks
 	do
-		case $arg in
-			crc32)
-				enable_checks_crc32=yes
-				AC_DEFINE([HAVE_CHECK_CRC32], 1,
-					[Define to 1 if CRC32 support
-					is enabled.])
-				;;
-			crc64)
-				enable_checks_crc64=yes
-				AC_DEFINE([HAVE_CHECK_CRC64], 1,
-					[Define to 1 if CRC64 support
-					is enabled.])
-				;;
-			sha256)
-				enable_checks_sha256=yes
-				AC_DEFINE([HAVE_CHECK_SHA256], 1,
-					[Define to 1 if SHA256 support
-					is enabled.])
-				;;
+		case $arg in m4_foreach([NAME], [SUPPORTED_CHECKS], [
+			NAME)
+				enable_check_[]NAME=yes
+				AC_DEFINE(HAVE_CHECK_[]m4_toupper(NAME), [1],
+				[Define to 1 if] NAME
+				[integrity check is enabled.])
+				;;])
 			*)
 				AC_MSG_RESULT([])
 				AC_MSG_ERROR([unknown integrity check: $arg])
@@ -286,41 +286,95 @@ fi
 if test "x$enable_checks_crc32" = xno ; then
 	AC_MSG_ERROR([For now, the CRC32 check must always be enabled.])
 fi
-AM_CONDITIONAL(COND_CHECK_CRC32, test "x$enable_checks_crc32" = xyes)
-AM_CONDITIONAL(COND_CHECK_CRC64, test "x$enable_checks_crc64" = xyes)
-AM_CONDITIONAL(COND_CHECK_SHA256, test "x$enable_checks_sha256" = xyes)
 
-# Assembler optimizations
+m4_foreach([NAME], [SUPPORTED_CHECKS],
+[AM_CONDITIONAL(COND_CHECK_[]m4_toupper(NAME), test "x$enable_check_[]NAME" = xyes)
+])dnl
+
+
+###########################
+# Assembler optimizations #
+###########################
+
 AC_MSG_CHECKING([if assembler optimizations should be used])
-AC_ARG_ENABLE(assembler, AC_HELP_STRING([--disable-assembler],
+AC_ARG_ENABLE([assembler], AC_HELP_STRING([--disable-assembler],
 		[Do not use assembler optimizations even if such exist
 		for the architecture.]),
-		[], [enable_assembler=yes])
+	[], [enable_assembler=yes])
 if test "x$enable_assembler" = xyes; then
 	case $host_cpu in
 		i?86)   enable_assembler=x86 ;;
+		x86_64) enable_assembler=x86_64 ;;
 		*)      enable_assembler=no ;;
 	esac
+	# Darwin has different ABI than GNU+Linux and Solaris,
+	# and the assembler code doesn't assemble.
+	case $host_os in
+		darwin*) enable_assembler=no ;;
+		*)       ;;
+	esac
 fi
 case $enable_assembler in
-	x86|no) ;;
+	x86)
+		AC_DEFINE([HAVE_ASM_X86], [1],
+			[Define to 1 if using x86 assembler optimizations.])
+		;;
+	x86_64)
+		AC_DEFINE([HAVE_ASM_X86_64], [1],
+			[Define to 1 if using x86_64 assembler optimizations.])
+		;;
+	no)
+		;;
 	*)
 		AC_MSG_RESULT([])
-		AC_MSG_ERROR([--enable-assembler accepts only \`yes', \`no', or \`x86'.])
+		AC_MSG_ERROR([--enable-assembler accepts only \`yes', \`no', \`x86', or \`x86_64'.])
 		;;
 esac
 AC_MSG_RESULT([$enable_assembler])
 AM_CONDITIONAL(COND_ASM_X86, test "x$enable_assembler" = xx86)
+AM_CONDITIONAL(COND_ASM_X86_64, test "x$enable_assembler" = xx86_64)
+
+
+################################
+# Fast unaligned memory access #
+################################
+
+AC_MSG_CHECKING([if unaligned memory access should be used])
+AC_ARG_ENABLE([unaligned-access], AC_HELP_STRING([--enable-unaligned-access],
+		[Enable if the system supports *fast* unaligned memory access
+		with 16-bit and 32-bit integers. By default, this is enabled
+		only on x86, x86_64, and big endian PowerPC.]),
+	[], [enable_unaligned_access=auto])
+if test "x$enable_unaligned_access" = xauto ; then
+	case $host_cpu in
+		i?86|x86_64|powerpc|powerpc64)
+			enable_unaligned_access=yes
+			;;
+		*)
+			enable_unaligned_access=no
+			;;
+	esac
+fi
+if test "x$enable_unaligned_access" = xyes ; then
+	AC_DEFINE([HAVE_FAST_UNALIGNED_ACCESS], [1], [Define to 1 if
+		the system supports fast unaligned memory access.])
+	AC_MSG_RESULT([yes])
+else
+	AC_MSG_RESULT([no])
+fi
+
+
+#####################
+# Size optimization #
+#####################
 
-# Size optimization
 AC_MSG_CHECKING([if small size is preferred over speed])
-AC_ARG_ENABLE(small, AC_HELP_STRING([--enable-small],
-		[Omit precomputed tables to make liblzma a few kilobytes
-		smaller. This will increase startup time of applications
-		slightly, because the tables need to be computed first.]),
-		[], [enable_small=no])
+AC_ARG_ENABLE([small], AC_HELP_STRING([--enable-small],
+		[Make liblzma smaller and a little slower.
+		This is disabled by default to optimize for speed.]),
+	[], [enable_small=no])
 if test "x$enable_small" = xyes; then
-	AC_DEFINE([HAVE_SMALL], 1, [Define to 1 if optimizing for size.])
+	AC_DEFINE([HAVE_SMALL], [1], [Define to 1 if optimizing for size.])
 elif test "x$enable_small" != xno; then
 	AC_MSG_RESULT([])
 	AC_MSG_ERROR([--enable-small accepts only \`yes' or \`no'])
@@ -328,32 +382,54 @@ fi
 AC_MSG_RESULT([$enable_small])
 AM_CONDITIONAL(COND_SMALL, test "x$enable_small" = xyes)
 
-echo
-echo "Initializing Automake:"
 
-# There's no C++ or Fortran in LZMA Utils:
-CXX=no
-F77=no
+#############
+# Threading #
+#############
 
-AM_INIT_AUTOMAKE
+AC_MSG_CHECKING([if threading support is wanted])
+AC_ARG_ENABLE([threads], AC_HELP_STRING([--disable-threads],
+		[Disable threading support.
+		This makes some things thread-unsafe.]),
+	[], [enable_threads=yes])
+if test "x$enable_threads" != xyes && test "x$enable_threads" != xno; then
+	AC_MSG_ERROR([--enable-threads accepts only \`yes' or \`no'])
+fi
+# We use the actual result a little later.
 
-AC_USE_SYSTEM_EXTENSIONS
 
 ###############################################################################
 # Checks for programs.
 ###############################################################################
 
-AM_PROG_CC_C_O
-AM_PROG_AS
+echo
+echo "Initializing Automake:"
+
+AM_INIT_AUTOMAKE([1.10 foreign tar-v7 filename-length-max=99])
 AC_PROG_LN_S
 
-echo
-echo "Threading support:"
-ACX_PTHREAD
-CC="$PTHREAD_CC"
+AC_PROG_CC_C99
+if test x$ac_cv_prog_cc_c99 = xno ; then
+	AC_MSG_ERROR([No C99 compiler was found.])
+fi
+
+AM_PROG_CC_C_O
+AM_PROG_AS
+AC_USE_SYSTEM_EXTENSIONS
+
+if test "x$enable_threads" = xyes; then
+	echo
+	echo "Threading support:"
+	ACX_PTHREAD
+	LIBS="$LIBS $PTHREAD_LIBS"
+	CFLAGS="$PTHREAD_CFLAGS $CFLAGS"
+	CC="$PTHREAD_CC"
+fi
 
 echo
 echo "Initializing Libtool:"
+CXX=no
+F77=no
 AC_PROG_LIBTOOL
 
 
@@ -380,33 +456,66 @@ AC_CHECK_HEADERS([fcntl.h limits.h sys/time.h],
 	[AC_MSG_ERROR([Required header file(s) are missing.])])
 
 # If any of these headers are missing, things should still work correctly:
-AC_CHECK_HEADERS([assert.h errno.h byteswap.h sys/param.h sys/sysctl.h],
+AC_CHECK_HEADERS([sys/param.h sys/sysctl.h byteswap.h],
 	[], [], [
 #ifdef HAVE_SYS_PARAM_H
 #	include <sys/param.h>
 #endif
 ])
 
+# Even if we have byteswap.h, we may lack the specific macros/functions.
+if test x$ac_cv_header_byteswap_h = xyes ; then
+	m4_foreach([FUNC], [bswap_16,bswap_32,bswap_64], [
+		AC_MSG_CHECKING([if FUNC is available])
+		AC_LINK_IFELSE([AC_LANG_SOURCE([
+#include <byteswap.h>
+int
+main(void)
+{
+	FUNC[](42);
+	return 0;
+}
+		])], [
+			AC_DEFINE(HAVE_[]m4_toupper(FUNC), [1],
+					[Define to 1 if] FUNC [is available.])
+			AC_MSG_RESULT([yes])
+		], [AC_MSG_RESULT([no])])
+
+	])dnl
+fi
+
 
 ###############################################################################
 # Checks for typedefs, structures, and compiler characteristics.
 ###############################################################################
 
+dnl We don't need these as long as we need a C99 compiler anyway.
+dnl AC_C_INLINE
+dnl AC_C_RESTRICT
+
 AC_HEADER_STDBOOL
-AC_C_INLINE
-AC_C_RESTRICT
+
+AC_TYPE_UINT8_T
+AC_TYPE_UINT16_T
+AC_TYPE_INT32_T
+AC_TYPE_UINT32_T
+AC_TYPE_INT64_T
+AC_TYPE_UINT64_T
+AC_TYPE_UINTPTR_T
+
+AC_CHECK_SIZEOF([size_t])
 
 # The command line tool can copy high resolution timestamps if such
 # information is availabe in struct stat. Otherwise one second accuracy
-# is used. Most systems seem to have st_xtim but BSDs have st_xtimespec.
-AC_CHECK_MEMBERS([struct stat.st_atim.tv_nsec, struct stat.st_mtim.tv_nsec,
-	struct stat.st_atimespec.tv_nsec, struct stat.st_mtimespec.tv_nsec])
+# is used.
+AC_CHECK_MEMBERS([
+	struct stat.st_atim.tv_nsec,
+	struct stat.st_atimespec.tv_nsec,
+	struct stat.st_atimensec,
+	struct stat.st_uatime,
+	struct stat.st_atim.st__tim.tv_nsec])
 
-# It is very unlikely that you want to build liblzma without
-# large file support.
 AC_SYS_LARGEFILE
-
-# At the moment, the only endian-dependent part should be the integrity checks.
 AC_C_BIGENDIAN
 
 
@@ -417,16 +526,15 @@ AC_C_BIGENDIAN
 # Gnulib replacements as needed
 gl_GETOPT
 
-# Functions that are not mandatory i.e. we have alternatives for them
-# or we can just drop some functionality:
-AC_CHECK_FUNCS([memcpy memmove memset futimes futimesat])
+# Find the best function to set timestamps.
+AC_CHECK_FUNCS([futimens futimes futimesat utimes utime], [break])
 
 # Check how to find out the amount of physical memory in the system. The
-# lzma command line tool uses this to automatically limits its memory usage.
+# xz command line tool uses this to automatically limit its memory usage.
 # - sysconf() gives all the needed info on GNU+Linux and Solaris.
 # - BSDs use sysctl().
 AC_MSG_CHECKING([how to detect the amount of physical memory])
-AC_COMPILE_IFELSE([
+AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
 #include <unistd.h>
 int
 main()
@@ -436,8 +544,8 @@ main()
 	i = sysconf(_SC_PHYS_PAGES);
 	return 0;
 }
-], [
-	AC_DEFINE([HAVE_PHYSMEM_SYSCONF], 1,
+]])], [
+	AC_DEFINE([HAVE_PHYSMEM_SYSCONF], [1],
 		[Define to 1 if the amount of physical memory can be detected
 		with sysconf(_SC_PAGESIZE) and sysconf(_SC_PHYS_PAGES).])
 	AC_MSG_RESULT([sysconf])
@@ -458,7 +566,7 @@ main()
 	return 0;
 }
 ]])], [
-	AC_DEFINE([HAVE_PHYSMEM_SYSCTL], 1,
+	AC_DEFINE([HAVE_PHYSMEM_SYSCTL], [1],
 		[Define to 1 if the amount of physical memory can be detected
 		with sysctl().])
 	AC_MSG_RESULT([sysctl])
@@ -470,7 +578,7 @@ main()
 # sysconf(_SC_NPROCESSORS_ONLN) works on most systems, except that BSDs
 # use sysctl().
 AC_MSG_CHECKING([how to detect the number of available CPU cores])
-AC_COMPILE_IFELSE([
+AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
 #include <unistd.h>
 int
 main()
@@ -479,8 +587,8 @@ main()
 	i = sysconf(_SC_NPROCESSORS_ONLN);
 	return 0;
 }
-], [
-	AC_DEFINE([HAVE_NCPU_SYSCONF], 1,
+]])], [
+	AC_DEFINE([HAVE_NCPU_SYSCONF], [1],
 		[Define to 1 if the number of available CPU cores can be
 		detected with sysconf(_SC_NPROCESSORS_ONLN).])
 	AC_MSG_RESULT([sysconf])
@@ -501,7 +609,7 @@ main()
 	return 0;
 }
 ]])], [
-	AC_DEFINE([HAVE_NCPU_SYSCTL], 1,
+	AC_DEFINE([HAVE_NCPU_SYSCTL], [1],
 		[Define to 1 if the number of available CPU cores can be
 		detected with sysctl().])
 	AC_MSG_RESULT([sysctl])
@@ -516,7 +624,7 @@ main()
 
 Wno_uninitialized=no
 
-if test -n "$GCC" ; then
+if test "x$GCC" = xyes ; then
 	echo
 	echo "GCC extensions:"
 	gl_VISIBILITY
@@ -534,7 +642,7 @@ if test -n "$GCC" ; then
 	AC_MSG_RESULT([$Wno_uninitialized])
 
 	# Enable as much warnings as possible. These commented warnings won't
-	# work for LZMA Utils though:
+	# work for this package though:
 	#   * -Wunreachable-code breaks several assert(0) cases, which are
 	#     backed up with "return LZMA_PROG_ERROR".
 	#   * -Wcast-qual would break various things where we need a non-const
@@ -545,13 +653,30 @@ if test -n "$GCC" ; then
 	#   * -Winline, -Wdisabled-optimization, -Wunsafe-loop-optimizations
 	#     don't seem so useful here; at least the last one gives some
 	#     warnings which are not bugs.
-	for NEW_FLAG in -Wextra -Wformat=2 -Winit-self -Wstrict-aliasing=2 \
-			-Wfloat-equal -Wshadow 	-Wpointer-arith \
-			-Wbad-function-cast -Wwrite-strings \
-			-Waggregate-return -Wstrict-prototypes \
-			-Wold-style-definition -Wmissing-prototypes \
-			-Wmissing-declarations -Wmissing-noreturn \
-			-Wredundant-decls
+	#
+	# The flags are in reverse order below so they end up in "beautiful"
+	# order on the actual command line.
+	for NEW_FLAG in \
+			-Wredundant-decls \
+			-Wmissing-noreturn \
+			-Wmissing-declarations \
+			-Wmissing-prototypes \
+			-Wold-style-definition \
+			-Wstrict-prototypes \
+			-Waggregate-return \
+			-Wlogical-op \
+			-Wwrite-strings \
+			-Wbad-function-cast \
+			-Wpointer-arith \
+			-Wshadow \
+			-Wundef \
+			-Wfloat-equal \
+			-Wstrict-aliasing \
+			-Wmissing-include-dirs \
+			-Winit-self \
+			-Wformat=2 \
+			-Wextra \
+			-Wall
 	do
 		AC_MSG_CHECKING([if $CC accepts $NEW_FLAG])
 		OLD_CFLAGS="$CFLAGS"
@@ -571,9 +696,6 @@ if test -n "$GCC" ; then
 	if test "x$enable_werror" = "xyes"; then
 		CFLAGS="-Werror $CFLAGS"
 	fi
-
-	# IIRC these work with all GCC versions that support -std=c99:
-	CFLAGS="-std=c99 -pedantic -Wall $CFLAGS"
 fi
 
 AM_CONDITIONAL([COND_WNO_UNINITIALIZED], test "x$Wno_uninitialized" = "xyes")
@@ -595,20 +717,22 @@ AC_CONFIG_FILES([
 	po/Makefile.in
 	lib/Makefile
 	src/Makefile
-	src/liblzma/lzma.pc
+	src/liblzma/liblzma.pc
 	src/liblzma/Makefile
 	src/liblzma/api/Makefile
 	src/liblzma/common/Makefile
 	src/liblzma/check/Makefile
+	src/liblzma/rangecoder/Makefile
 	src/liblzma/lz/Makefile
 	src/liblzma/lzma/Makefile
-	src/liblzma/simple/Makefile
 	src/liblzma/subblock/Makefile
-	src/liblzma/rangecoder/Makefile
-	src/lzma/Makefile
-	src/lzmadec/Makefile
+	src/liblzma/delta/Makefile
+	src/liblzma/simple/Makefile
+	src/xz/Makefile
+	src/xzdec/Makefile
 	src/scripts/Makefile
 	tests/Makefile
+	debug/Makefile
 ])
 
 AC_OUTPUT

debug/Makefile.am

@@ -0,0 +1,36 @@
+##
+##  Copyright (C) 2008 Lasse Collin
+##
+##  This library is free software; you can redistribute it and/or
+##  modify it under the terms of the GNU Lesser General Public
+##  License as published by the Free Software Foundation; either
+##  version 2.1 of the License, or (at your option) any later version.
+##
+##  This library is distributed in the hope that it will be useful,
+##  but WITHOUT ANY WARRANTY; without even the implied warranty of
+##  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+##  Lesser General Public License for more details.
+##
+
+noinst_PROGRAMS = \
+	repeat \
+	sync_flush \
+	full_flush \
+	memusage \
+	crc32 \
+	known_sizes \
+	hex2bin
+
+AM_CPPFLAGS = \
+	-I@top_srcdir@/src/common \
+	-I@top_srcdir@/src/liblzma/api
+
+AM_LDFLAGS = -static
+
+LDADD = \
+	@top_builddir@/src/liblzma/liblzma.la \
+	@LTLIBINTL@
+
+if COND_GNULIB
+LDADD += @top_builddir@/lib/libgnu.a
+endif

debug/README

@@ -0,0 +1,17 @@
+
+Debug tools
+-----------
+
+    This directory contains a few tiny programs that may be helpful when
+    debugging LZMA Utils.
+
+    These tools are not meant to be installed. Often one needs to edit
+    the source code a little to make the programs do the wanted things.
+    If you don't know how these programs could help you, it is likely
+    that they really are useless to you.
+
+    These aren't intended to be used as example programs. They take some
+    shortcuts here and there, which correct programs should not do. Many
+    possible errors (especially I/O errors) are ignored. Don't report
+    bugs or send patches to fix this kind of bugs.
+

debug/crc32.c

@@ -0,0 +1,45 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       crc32.c
+/// \brief      Primitive CRC32 calculation tool
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+
+
+int
+main(void)
+{
+	uint32_t crc = 0;
+
+	do {
+		uint8_t buf[BUFSIZ];
+		const size_t size = fread(buf, 1, sizeof(buf), stdin);
+		crc = lzma_crc32(buf, size, crc);
+	} while (!ferror(stdin) && !feof(stdin));
+
+	//printf("%08" PRIX32 "\n", crc);
+
+	// I want it little endian so it's easy to work with hex editor.
+	printf("%02" PRIX32 " ", crc & 0xFF);
+	printf("%02" PRIX32 " ", (crc >> 8) & 0xFF);
+	printf("%02" PRIX32 " ", (crc >> 16) & 0xFF);
+	printf("%02" PRIX32 " ", crc >> 24);
+	printf("\n");
+
+	return 0;
+}

debug/full_flush.c

@@ -0,0 +1,108 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       full_flush.c
+/// \brief      Encode files using LZMA_FULL_FLUSH
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+
+
+static lzma_stream strm = LZMA_STREAM_INIT;
+static FILE *file_in;
+
+
+static void
+encode(size_t size, lzma_action action)
+{
+	static const size_t CHUNK = 64;
+	uint8_t in[CHUNK];
+	uint8_t out[CHUNK];
+	lzma_ret ret;
+
+	do {
+		if (strm.avail_in == 0 && size > 0) {
+			const size_t amount = MIN(size, CHUNK);
+			strm.avail_in = fread(in, 1, amount, file_in);
+			strm.next_in = in;
+			size -= amount; // Intentionally not using avail_in.
+		}
+
+		strm.next_out = out;
+		strm.avail_out = CHUNK;
+
+		ret = lzma_code(&strm, size == 0 ? action : LZMA_RUN);
+
+		if (ret != LZMA_OK && ret != LZMA_STREAM_END) {
+			fprintf(stderr, "%s:%u: %s: ret == %d\n",
+					__FILE__, __LINE__, __func__, ret);
+			exit(1);
+		}
+
+		fwrite(out, 1, CHUNK - strm.avail_out, stdout);
+
+	} while (size > 0 || strm.avail_out == 0);
+
+	if ((action == LZMA_RUN && ret != LZMA_OK)
+			|| (action != LZMA_RUN && ret != LZMA_STREAM_END)) {
+		fprintf(stderr, "%s:%u: %s: ret == %d\n",
+				__FILE__, __LINE__, __func__, ret);
+		exit(1);
+	}
+}
+
+
+int
+main(int argc, char **argv)
+{
+	file_in = argc > 1 ? fopen(argv[1], "rb") : stdin;
+
+
+	// Config
+	lzma_options_lzma opt_lzma;
+	if (lzma_lzma_preset(&opt_lzma, 1)) {
+		fprintf(stderr, "preset failed\n");
+		exit(1);
+	}
+	lzma_filter filters[LZMA_FILTERS_MAX + 1];
+	filters[0].id = LZMA_FILTER_LZMA2;
+	filters[0].options = &opt_lzma;
+	filters[1].id = LZMA_VLI_UNKNOWN;
+
+	// Init
+	if (lzma_stream_encoder(&strm, filters, LZMA_CHECK_CRC32) != LZMA_OK) {
+		fprintf(stderr, "init failed\n");
+		exit(1);
+	}
+
+// 	if (lzma_easy_encoder(&strm, 1)) {
+// 		fprintf(stderr, "init failed\n");
+// 		exit(1);
+// 	}
+
+	// Encoding
+	encode(0, LZMA_FULL_FLUSH);
+	encode(6, LZMA_FULL_FLUSH);
+	encode(0, LZMA_FULL_FLUSH);
+	encode(7, LZMA_FULL_FLUSH);
+	encode(0, LZMA_FULL_FLUSH);
+	encode(0, LZMA_FINISH);
+
+	// Clean up
+	lzma_end(&strm);
+
+	return 0;
+}

debug/hex2bin.c

@@ -0,0 +1,54 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       hex2bin.c
+/// \brief      Converts hexadecimal input strings to binary
+//
+//  This code has been put into the public domain.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+#include <ctype.h>
+
+
+static int
+getbin(int x)
+{
+	if (x >= '0' && x <= '9')
+		return x - '0';
+
+	if (x >= 'A' && x <= 'F')
+		return x - 'A' + 10;
+
+	return x - 'a' + 10;
+}
+
+
+int
+main(void)
+{
+	while (true) {
+		int byte = getchar();
+		if (byte == EOF)
+			return 0;
+		if (!isxdigit(byte))
+			continue;
+
+		const int digit = getchar();
+		if (digit == EOF || !isxdigit(digit)) {
+			fprintf(stderr, "Invalid input\n");
+			return 1;
+		}
+
+		byte = (getbin(byte) << 4) | getbin(digit);
+		if (putchar(byte) == EOF) {
+			perror(NULL);
+			return 1;
+		}
+	}
+}

debug/known_sizes.c

@@ -0,0 +1,135 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       known_sizes.c
+/// \brief      Encodes .lzma Stream with sizes known in Block Header
+///
+/// The input file is encoded in RAM, and the known Compressed Size
+/// and/or Uncompressed Size values are stored in the Block Header.
+/// As of writing there's no such Stream encoder in liblzma.
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/unistd.h>
+#include <stdio.h>
+
+
+// Support file sizes up to 1 MiB. We use this for output space too, so files
+// close to 1 MiB had better compress at least a little or we have a buffer
+// overflow.
+#define BUFFER_SIZE (1U << 20)
+
+
+int
+main(void)
+{
+	// Allocate the buffers.
+	uint8_t *in = malloc(BUFFER_SIZE);
+	uint8_t *out = malloc(BUFFER_SIZE);
+	if (in == NULL || out == NULL)
+		return 1;
+
+	// Fill the input buffer.
+	const size_t in_size = fread(in, 1, BUFFER_SIZE, stdin);
+
+	// Filter setup
+	lzma_options_lzma opt_lzma;
+	if (lzma_lzma_preset(&opt_lzma, 1))
+		return 1;
+	
+	lzma_filter filters[] = {
+		{
+			.id = LZMA_FILTER_LZMA2,
+			.options = &opt_lzma
+		},
+		{
+			.id = LZMA_VLI_UNKNOWN
+		}
+	};
+
+	lzma_block block = {
+		.check = LZMA_CHECK_CRC32,
+		.compressed_size = BUFFER_SIZE, // Worst case reserve
+		.uncompressed_size = in_size,
+		.filters = filters,
+	};
+
+	lzma_stream strm = LZMA_STREAM_INIT;
+	if (lzma_block_encoder(&strm, &block) != LZMA_OK)
+		return 1;
+
+	// Reserve space for Stream Header and Block Header. We need to
+	// calculate the size of the Block Header first.
+	if (lzma_block_header_size(&block) != LZMA_OK)
+		return 1;
+
+	size_t out_size = LZMA_STREAM_HEADER_SIZE + block.header_size;
+
+	strm.next_in = in;
+	strm.avail_in = in_size;
+	strm.next_out = out + out_size;
+	strm.avail_out = BUFFER_SIZE - out_size;
+
+	if (lzma_code(&strm, LZMA_FINISH) != LZMA_STREAM_END)
+		return 1;
+
+	out_size += strm.total_out;
+
+	if (lzma_block_header_encode(&block, out + LZMA_STREAM_HEADER_SIZE)
+			!= LZMA_OK)
+		return 1;
+
+	lzma_index *idx = lzma_index_init(NULL, NULL);
+	if (idx == NULL)
+		return 1;
+
+	if (lzma_index_append(idx, NULL, block.header_size + strm.total_out,
+			strm.total_in) != LZMA_OK)
+		return 1;
+
+	if (lzma_index_encoder(&strm, idx) != LZMA_OK)
+		return 1;
+
+	if (lzma_code(&strm, LZMA_RUN) != LZMA_STREAM_END)
+		return 1;
+
+	out_size += strm.total_out;
+
+	lzma_end(&strm);
+
+	lzma_index_end(idx, NULL);
+
+	// Encode the Stream Header and Stream Footer. backwards_size is
+	// needed only for the Stream Footer.
+	lzma_stream_flags sf = {
+		.backward_size = strm.total_out,
+		.check = block.check,
+	};
+
+	if (lzma_stream_header_encode(&sf, out) != LZMA_OK)
+		return 1;
+
+	if (lzma_stream_footer_encode(&sf, out + out_size) != LZMA_OK)
+		return 1;
+
+	out_size += LZMA_STREAM_HEADER_SIZE;
+
+	// Write out the file.
+	fwrite(out, 1, out_size, stdout);
+
+	return 0;
+}

debug/memusage.c

@@ -0,0 +1,55 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       memusage.c
+/// \brief      Calculates memory usage using lzma_memory_usage()
+///
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+
+int
+main(void)
+{
+	lzma_options_lzma lzma = {
+		.dict_size = (1U << 30) + (1U << 29),
+		.lc = 3,
+		.lp = 0,
+		.pb = 2,
+		.preset_dict = NULL,
+		.preset_dict_size = 0,
+		.mode = LZMA_MODE_NORMAL,
+		.nice_len = 48,
+		.mf = LZMA_MF_BT4,
+		.depth = 0,
+	};
+
+/*
+	lzma_options_filter filters[] = {
+		{ LZMA_FILTER_LZMA1,
+			(lzma_options_lzma *)&lzma_preset_lzma[6 - 1] },
+		{ UINT64_MAX, NULL }
+	};
+*/
+	lzma_filter filters[] = {
+		{ LZMA_FILTER_LZMA1, &lzma },
+		{ UINT64_MAX, NULL }
+	};
+
+	printf("Encoder: %10" PRIu64 " B\n", lzma_memusage_encoder(filters));
+	printf("Decoder: %10" PRIu64 " B\n", lzma_memusage_decoder(filters));
+
+	return 0;
+}

debug/repeat.c

@@ -0,0 +1,43 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       repeat.c
+/// \brief      Repeats given string given times
+///
+/// This program can be useful when debugging run-length encoder in
+/// the Subblock filter, especially the condition when repeat count
+/// doesn't fit into 28-bit integer.
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+
+
+int
+main(int argc, char **argv)
+{
+	if (argc != 3) {
+		fprintf(stderr, "Usage: %s COUNT STRING\n", argv[0]);
+		exit(1);
+	}
+
+	unsigned long long count = strtoull(argv[1], NULL, 10);
+	const size_t size = strlen(argv[2]);
+
+	while (count-- != 0)
+		fwrite(argv[2], 1, size, stdout);
+
+	return !!(ferror(stdout) || fclose(stdout));
+}

debug/sync_flush.c

@@ -0,0 +1,139 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       sync_flush.c
+/// \brief      Encode files using LZMA_SYNC_FLUSH
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+#include <stdio.h>
+
+
+static lzma_stream strm = LZMA_STREAM_INIT;
+static FILE *file_in;
+
+
+static void
+encode(size_t size, lzma_action action)
+{
+	static const size_t CHUNK = 64;
+	uint8_t in[CHUNK];
+	uint8_t out[CHUNK];
+	lzma_ret ret;
+
+	do {
+		if (strm.avail_in == 0 && size > 0) {
+			const size_t amount = MIN(size, CHUNK);
+			strm.avail_in = fread(in, 1, amount, file_in);
+			strm.next_in = in;
+			size -= amount; // Intentionally not using avail_in.
+		}
+
+		strm.next_out = out;
+		strm.avail_out = CHUNK;
+
+		ret = lzma_code(&strm, size == 0 ? action : LZMA_RUN);
+
+		if (ret != LZMA_OK && ret != LZMA_STREAM_END) {
+			fprintf(stderr, "%s:%u: %s: ret == %d\n",
+					__FILE__, __LINE__, __func__, ret);
+			exit(1);
+		}
+
+		fwrite(out, 1, CHUNK - strm.avail_out, stdout);
+
+	} while (size > 0 || strm.avail_out == 0);
+
+	if ((action == LZMA_RUN && ret != LZMA_OK)
+			|| (action != LZMA_RUN && ret != LZMA_STREAM_END)) {
+		fprintf(stderr, "%s:%u: %s: ret == %d\n",
+				__FILE__, __LINE__, __func__, ret);
+		exit(1);
+	}
+}
+
+
+int
+main(int argc, char **argv)
+{
+	file_in = argc > 1 ? fopen(argv[1], "rb") : stdin;
+
+	// Config
+	lzma_options_lzma opt_lzma = {
+		.dict_size = 1U << 16,
+		.lc = LZMA_LC_DEFAULT,
+		.lp = LZMA_LP_DEFAULT,
+		.pb = LZMA_PB_DEFAULT,
+		.preset_dict = NULL,
+		.persistent = true,
+		.mode = LZMA_MODE_NORMAL,
+		.nice_len = 32,
+		.mf = LZMA_MF_HC3,
+		.depth = 0,
+	};
+
+	lzma_options_delta opt_delta = {
+		.dist = 16
+	};
+
+	lzma_options_subblock opt_subblock = {
+		.allow_subfilters = true,
+		.alignment = 8, // LZMA_SUBBLOCK_ALIGNMENT_DEFAULT,
+		.subblock_data_size = LZMA_SUBBLOCK_DATA_SIZE_DEFAULT,
+		.rle = 1, // LZMA_SUBBLOCK_RLE_OFF,
+		.subfilter_mode = LZMA_SUBFILTER_SET,
+	};
+	opt_subblock.subfilter_options.id = LZMA_FILTER_LZMA1;
+	opt_subblock.subfilter_options.options = &opt_lzma;
+	opt_subblock.subfilter_options.id = LZMA_FILTER_DELTA;
+	opt_subblock.subfilter_options.options = &opt_delta;
+
+	lzma_filter filters[LZMA_FILTERS_MAX + 1];
+	filters[0].id = LZMA_FILTER_LZMA2;
+	filters[0].options = &opt_lzma;
+	filters[1].id = LZMA_VLI_UNKNOWN;
+
+	// Init
+	if (lzma_stream_encoder(&strm, filters, LZMA_CHECK_CRC32) != LZMA_OK) {
+		fprintf(stderr, "init failed\n");
+		exit(1);
+	}
+
+	// Encoding
+
+	encode(0, LZMA_SYNC_FLUSH);
+	encode(6, LZMA_SYNC_FLUSH);
+	encode(0, LZMA_SYNC_FLUSH);
+	encode(7, LZMA_SYNC_FLUSH);
+	encode(0, LZMA_SYNC_FLUSH);
+	encode(0, LZMA_FINISH);
+/*
+	encode(53, LZMA_SYNC_FLUSH);
+// 	opt_lzma.literal_context_bits = 2;
+// 	opt_lzma.literal_pos_bits = 1;
+// 	opt_lzma.pos_bits = 0;
+	encode(404, LZMA_FINISH);
+*/
+	// Clean up
+	lzma_end(&strm);
+
+	return 0;
+
+	// Prevent useless warnings so we don't need to have special CFLAGS
+	// to disable -Werror.
+	(void)opt_lzma;
+	(void)opt_subblock;
+	(void)opt_delta;
+}

doc/faq.txt

@@ -80,8 +80,8 @@ Q:  Which file formats are supported by LZMA Utils?
 A:  Even when the raw LZMA stream is always the same, it can be wrapped
     in different container formats. The preferred format is the new .lzma
     format. It has magic bytes (the first six bytes: 0xFF 'L' 'Z' 'M'
-    'A' 0x00). The format supports chaining up to seven filters filters,
-    splitting data to multiple blocks for easier multi-threading and rough
+    'A' 0x00). The format supports chaining up to seven filters, splitting
+    data to multiple blocks for easier multi-threading and rough
     random-access reading. The file integrity is verified using CRC32,
     CRC64, or SHA256, and by verifying the uncompressed size of the file.
 

doc/liblzma-advanced.txt

@@ -247,7 +247,7 @@ y.x.1. Single-Block Stream
     this prevents it from producing too much output in case of (possibly
     intentionally) corrupt file.
 
-    Calculate the the start offset of the Stream:
+    Calculate the start offset of the Stream:
 
         backward_offset - backward_size - LZMA_STREAM_HEADER_SIZE
 

doc/liblzma-intro.txt

@@ -5,8 +5,14 @@ Introduction to liblzma
 Writing applications to work with liblzma
 
     liblzma API is split in several subheaders to improve readability and
-    maintainance. The subheaders must not be #included directly; simply
-    use `#include <lzma.h>' instead.
+    maintainance. The subheaders must not be #included directly. lzma.h
+    requires that certain integer types and macros are available when
+    the header is #included. On systems that have inttypes.h that conforms
+    to C99, the following will work:
+
+        #include <sys/types.h>
+        #include <inttypes.h>
+        #include <lzma.h>
 
     Those who have used zlib should find liblzma's API easy to use.
     To developers who haven't used zlib before, I recommend learning

doc/liblzma-security.txt

@@ -86,8 +86,8 @@ Using liblzma securely
 
     The simplest solution is to use setrlimit() if the kernel supports
     RLIMIT_AS, which limits the memory usage of the whole process.
-    For more portable and fine-grained limitting, you can use
-    memory limitter functions found from <lzma/memlimit.h>.
+    For more portable and fine-grained limiting, you can use
+    memory limiter functions found from <lzma/memlimit.h>.
 
 
 1.2.1. Encoder
@@ -118,24 +118,24 @@ Using liblzma securely
 
 1.2.2. Decoder
 
-    A single-threaded decoder should simply use a memory limitter and
+    A single-threaded decoder should simply use a memory limiter and
     indicate an error if it runs out of memory.
 
-    Memory-limitting with multi-threaded decoding is tricky. The simple
+    Memory-limiting with multi-threaded decoding is tricky. The simple
     solution is to divide the maximum allowed memory usage with the
     maximum allowed threads, and give each Block decoder their own
-    independent lzma_memory_limitter. The drawback is that if one Block
+    independent lzma_memory_limiter. The drawback is that if one Block
     needs notably more RAM than any other Block, the decoder will run out
     of memory when in reality there would be plenty of free RAM.
 
-    An attractive alternative would be using shared lzma_memory_limitter.
+    An attractive alternative would be using shared lzma_memory_limiter.
     Depending on the application and the expected type of input, this may
     either be the best solution or a source of hard-to-repeat problems.
     Consider the following requirements:
-      - You use at maximum of n threads.
+      - You use a maximum of n threads.
       - x(i) is the decoder memory requirements of the Block number i
         in an expected input Stream.
-      - The memory limitter is set to higher value than the sum of n
+      - The memory limiter is set to higher value than the sum of n
         highest values x(i).
 
     (If you are better at explaining the above conditions, please
@@ -150,7 +150,7 @@ Using liblzma securely
     Most .lzma files have all the Blocks encoded with identical settings,
     or at least the memory usage won't vary dramatically. That's why most
     multi-threaded decoders probably want to use the simple "separate
-    lzma_memory_limitter for each thread" solution, possibly fallbacking
+    lzma_memory_limiter for each thread" solution, possibly falling back
     to single-threaded mode in case the per-thread memory limits aren't
     enough in multi-threaded mode.
 
@@ -206,7 +206,7 @@ FIXME: Memory usage of Stream info.
     creating a denial of service like piping decoded a Data Block to
     another process would do.
 
-    At first it would seem that using a memory limitter would prevent
+    At first it would seem that using a memory limiter would prevent
     this issue as a side effect. But it does so only if the application
     requests liblzma to allocate the Extra Records and provide them to
     the application. If Extra Records aren't requested, they aren't

extra/7z2lzma/7z2lzma.bash

@@ -0,0 +1,114 @@
+#!/bin/bash
+#
+#############################################################################
+#
+# 7z2lzma.bash is very primitive .7z to .lzma converter. The input file must
+# have exactly one LZMA compressed stream, which has been created with the
+# default lc, lp, and pb values. The CRC32 in the .7z archive is not checked,
+# and the script may seem to succeed while it actually created a corrupt .lzma
+# file. You should always try uncompressing both the original .7z and the
+# created .lzma and compare that the output is identical.
+#
+# This script requires basic GNU tools and 7z or 7za tool from p7zip.
+#
+# Last modified: 2008-09-27 11:05+0300
+#
+#############################################################################
+#
+# Copyright (C) 2008 Lasse Collin <lasse.collin@tukaani.org>
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be
+# included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+#############################################################################
+
+# You can use 7z or 7za, both will work.
+SEVENZIP=7za
+
+if [ $# != 2 -o -z "$1" -o -z "$2" ]; then
+	echo "Usage: $0 input.7z output.lzma"
+	exit 1
+fi
+
+# Converts an integer variable to little endian binary integer.
+int2bin()
+{
+	local LEN=$1
+	local NUM=$2
+	local HEX=(0 1 2 3 4 5 6 7 8 9 A B C D E F)
+	local I
+	for ((I=0; I < "$LEN"; ++I)); do
+		printf "\\x${HEX[(NUM >> 4) & 0x0F]}${HEX[NUM & 0x0F]}"
+		NUM=$((NUM >> 8))
+	done
+}
+
+# Make sure we get possible errors from pipes.
+set -o pipefail
+
+# Get information about the input file. At least older 7z and 7za versions
+# may return with zero exit status even when an error occurred, so check
+# if the output has any lines beginning with "Error".
+INFO=$("$SEVENZIP" l -slt "$1")
+if [ $? != 0 ] || printf '%s\n' "$INFO" | grep -q ^Error; then
+	printf '%s\n' "$INFO"
+	exit 1
+fi
+
+# Check if the input file has more than one compressed block.
+if printf '%s\n' "$INFO" | grep -q '^Block = 1'; then
+	echo "Cannot convert, because the input file has more than"
+	echo "one compressed block."
+	exit 1
+fi
+
+# Get copmressed, uncompressed, and dictionary size.
+CSIZE=$(printf '%s\n' "$INFO" | sed -rn 's|^Packed Size = ([0-9]+$)|\1|p')
+USIZE=$(printf '%s\n' "$INFO" | sed -rn 's|^Size = ([0-9]+$)|\1|p')
+DICT=$(printf '%s\n' "$INFO" | sed -rn 's|^Method = LZMA:([0-9]+)$|\1|p')
+
+if [ -z "$CSIZE" -o -z "$USIZE" -o -z "$DICT" ]; then
+	echo "Parsing output of $SEVENZIP failed. Maybe the file uses some"
+	echo "other compression method than plain LZMA."
+	exit 1
+fi
+
+# The following assumes that the default lc, lp, and pb settings were used.
+# Otherwise the output will be corrupt.
+printf '\x5D' > "$2"
+
+# Dictionary size
+int2bin 4 "$((1 << DICT))" >> "$2"
+
+# Uncompressed size
+int2bin 8 "$USIZE" >> "$2"
+
+# Copy the actual compressed data. Using multiple dd commands to avoid
+# copying large amount of data with one-byte block size, which would be
+# annoyingly slow.
+BS=8192
+BIGSIZE=$((CSIZE / BS))
+CSIZE=$((CSIZE % BS))
+{
+	dd of=/dev/null bs=32 count=1 \
+		&& dd bs="$BS" count="$BIGSIZE" \
+		&& dd bs=1 count="$CSIZE"
+} < "$1" >> "$2"
+
+exit $?

lib/Makefile.am

@@ -1,6 +1,5 @@
 ##
 ##  Copyright (C) 2004-2007 Free Software Foundation, Inc.
-##  Copyright (C) 2007 Lasse Collin
 ##
 ##  This program is free software; you can redistribute it and/or modify
 ##  it under the terms of the GNU General Public License as published by
@@ -13,28 +12,21 @@
 ##  GNU General Public License for more details.
 ##
 
-## Not using gnulib-tool, at least for now. Less mess this way.
+## Not using gnulib-tool, at least for now. It is likely that we won't
+## need anything else from Gnulib than getopt_long().
 
-## We need two builds of libgnu: one with NLS and one without.
-## This is because lzma uses NLS but lzmadec doesn't, while
-## both need GNU getopt_long().
-noinst_LIBRARIES = libgnu.a libgnu_nls.a
+noinst_LIBRARIES = libgnu.a
 
 libgnu_a_SOURCES =
 libgnu_a_DEPENDENCIES = $(LIBOBJS)
 libgnu_a_LIBADD = $(LIBOBJS)
-libgnu_a_CPPFLAGS = -DDISABLE_NLS=1
 
-libgnu_nls_a_SOURCES =
-libgnu_nls_a_DEPENDENCIES = $(LIBOBJS)
-libgnu_nls_a_LIBADD = $(LIBOBJS)
-
-EXTRA_DIST = gettext.h getopt_.h getopt.c getopt1.c getopt_int.h
+EXTRA_DIST = getopt.in.h getopt.c getopt1.c getopt_int.h
 BUILT_SOURCES = $(GETOPT_H)
 MOSTLYCLEANFILES = getopt.h getopt.h-t
 
-getopt.h: getopt_.h
+getopt.h: getopt.in.h
 	{ echo '/* DO NOT EDIT! GENERATED AUTOMATICALLY! */'; \
-	  cat $(srcdir)/getopt_.h; \
+	  cat $(srcdir)/getopt.in.h; \
 	} > $@-t
 	mv -f $@-t $@

lib/getopt.c

@@ -7,16 +7,16 @@
    This file is part of the GNU C Library.
 
    This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation; either version 2, or (at your option)
+   it under the terms of the GNU Lesser General Public License as published by
+   the Free Software Foundation; either version 2.1, or (at your option)
    any later version.
 
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
+   GNU Lesser General Public License for more details.
 
-   You should have received a copy of the GNU General Public License along
+   You should have received a copy of the GNU Lesser General Public License along
    with this program; if not, write to the Free Software Foundation,
    Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
 
@@ -35,12 +35,18 @@
 # include <unixlib.h>
 #endif
 
+/* Completely disable NLS for getopt. We won't include translations for it
+   anyway. If the system lacks getopt_long, missing translations probably
+   aren't a problem.  */
+/*
 #ifdef _LIBC
 # include <libintl.h>
 #else
 # include "gettext.h"
 # define _(msgid) gettext (msgid)
 #endif
+*/
+#define _(msgid) (msgid)
 
 #if defined _LIBC && defined USE_IN_LIBIO
 # include <wchar.h>

lib/getopt.in.h

@@ -4,16 +4,16 @@
    This file is part of the GNU C Library.
 
    This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation; either version 2, or (at your option)
+   it under the terms of the GNU Lesser General Public License as published by
+   the Free Software Foundation; either version 2.1, or (at your option)
    any later version.
 
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
+   GNU Lesser General Public License for more details.
 
-   You should have received a copy of the GNU General Public License along
+   You should have received a copy of the GNU Lesser General Public License along
    with this program; if not, write to the Free Software Foundation,
    Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
 

lib/getopt1.c

@@ -4,16 +4,16 @@
    This file is part of the GNU C Library.
 
    This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation; either version 2, or (at your option)
+   it under the terms of the GNU Lesser General Public License as published by
+   the Free Software Foundation; either version 2.1, or (at your option)
    any later version.
 
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
+   GNU Lesser General Public License for more details.
 
-   You should have received a copy of the GNU General Public License along
+   You should have received a copy of the GNU Lesser General Public License along
    with this program; if not, write to the Free Software Foundation,
    Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
 

lib/gettext.h

@@ -1,240 +0,0 @@
-/* Convenience header for conditional use of GNU <libintl.h>.
-   Copyright (C) 1995-1998, 2000-2002, 2004-2006 Free Software Foundation, Inc.
-
-   This program is free software; you can redistribute it and/or modify it
-   under the terms of the GNU Library General Public License as published
-   by the Free Software Foundation; either version 2, or (at your option)
-   any later version.
-
-   This program is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-   Library General Public License for more details.
-
-   You should have received a copy of the GNU Library General Public
-   License along with this program; if not, write to the Free Software
-   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
-   USA.  */
-
-#ifndef _LIBGETTEXT_H
-#define _LIBGETTEXT_H 1
-
-/* NLS can be disabled through the configure --disable-nls option.
- *
- * Extra hack in LZMA Utils: if DISABLE_NLS is defined, NLS is disabled
- * even if ENABLE_NLS is true. See Makefile.am for more information.
- */
-#if ENABLE_NLS && !defined(DISABLE_NLS)
-
-/* Get declarations of GNU message catalog functions.  */
-# include <libintl.h>
-
-/* You can set the DEFAULT_TEXT_DOMAIN macro to specify the domain used by
-   the gettext() and ngettext() macros.  This is an alternative to calling
-   textdomain(), and is useful for libraries.  */
-# ifdef DEFAULT_TEXT_DOMAIN
-#  undef gettext
-#  define gettext(Msgid) \
-     dgettext (DEFAULT_TEXT_DOMAIN, Msgid)
-#  undef ngettext
-#  define ngettext(Msgid1, Msgid2, N) \
-     dngettext (DEFAULT_TEXT_DOMAIN, Msgid1, Msgid2, N)
-# endif
-
-#else
-
-/* Solaris /usr/include/locale.h includes /usr/include/libintl.h, which
-   chokes if dcgettext is defined as a macro.  So include it now, to make
-   later inclusions of <locale.h> a NOP.  We don't include <libintl.h>
-   as well because people using "gettext.h" will not include <libintl.h>,
-   and also including <libintl.h> would fail on SunOS 4, whereas <locale.h>
-   is OK.  */
-#if defined(__sun)
-# include <locale.h>
-#endif
-
-/* Many header files from the libstdc++ coming with g++ 3.3 or newer include
-   <libintl.h>, which chokes if dcgettext is defined as a macro.  So include
-   it now, to make later inclusions of <libintl.h> a NOP.  */
-#if defined(__cplusplus) && defined(__GNUG__) && (__GNUC__ >= 3)
-# include <cstdlib>
-# if (__GLIBC__ >= 2) || _GLIBCXX_HAVE_LIBINTL_H
-#  include <libintl.h>
-# endif
-#endif
-
-/* Disabled NLS.
-   The casts to 'const char *' serve the purpose of producing warnings
-   for invalid uses of the value returned from these functions.
-   On pre-ANSI systems without 'const', the config.h file is supposed to
-   contain "#define const".  */
-# define gettext(Msgid) ((const char *) (Msgid))
-# define dgettext(Domainname, Msgid) ((const char *) (Msgid))
-# define dcgettext(Domainname, Msgid, Category) ((const char *) (Msgid))
-# define ngettext(Msgid1, Msgid2, N) \
-    ((N) == 1 ? (const char *) (Msgid1) : (const char *) (Msgid2))
-# define dngettext(Domainname, Msgid1, Msgid2, N) \
-    ((N) == 1 ? (const char *) (Msgid1) : (const char *) (Msgid2))
-# define dcngettext(Domainname, Msgid1, Msgid2, N, Category) \
-    ((N) == 1 ? (const char *) (Msgid1) : (const char *) (Msgid2))
-# define textdomain(Domainname) ((const char *) (Domainname))
-# define bindtextdomain(Domainname, Dirname) ((const char *) (Dirname))
-# define bind_textdomain_codeset(Domainname, Codeset) ((const char *) (Codeset))
-
-#endif
-
-/* A pseudo function call that serves as a marker for the automated
-   extraction of messages, but does not call gettext().  The run-time
-   translation is done at a different place in the code.
-   The argument, String, should be a literal string.  Concatenated strings
-   and other string expressions won't work.
-   The macro's expansion is not parenthesized, so that it is suitable as
-   initializer for static 'char[]' or 'const char[]' variables.  */
-#define gettext_noop(String) String
-
-/* The separator between msgctxt and msgid in a .mo file.  */
-#define GETTEXT_CONTEXT_GLUE "\004"
-
-/* Pseudo function calls, taking a MSGCTXT and a MSGID instead of just a
-   MSGID.  MSGCTXT and MSGID must be string literals.  MSGCTXT should be
-   short and rarely need to change.
-   The letter 'p' stands for 'particular' or 'special'.  */
-#ifdef DEFAULT_TEXT_DOMAIN
-# define pgettext(Msgctxt, Msgid) \
-   pgettext_aux (DEFAULT_TEXT_DOMAIN, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, LC_MESSAGES)
-#else
-# define pgettext(Msgctxt, Msgid) \
-   pgettext_aux (NULL, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, LC_MESSAGES)
-#endif
-#define dpgettext(Domainname, Msgctxt, Msgid) \
-  pgettext_aux (Domainname, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, LC_MESSAGES)
-#define dcpgettext(Domainname, Msgctxt, Msgid, Category) \
-  pgettext_aux (Domainname, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, Category)
-#ifdef DEFAULT_TEXT_DOMAIN
-# define npgettext(Msgctxt, Msgid, MsgidPlural, N) \
-   npgettext_aux (DEFAULT_TEXT_DOMAIN, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, MsgidPlural, N, LC_MESSAGES)
-#else
-# define npgettext(Msgctxt, Msgid, MsgidPlural, N) \
-   npgettext_aux (NULL, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, MsgidPlural, N, LC_MESSAGES)
-#endif
-#define dnpgettext(Domainname, Msgctxt, Msgid, MsgidPlural, N) \
-  npgettext_aux (Domainname, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, MsgidPlural, N, LC_MESSAGES)
-#define dcnpgettext(Domainname, Msgctxt, Msgid, MsgidPlural, N, Category) \
-  npgettext_aux (Domainname, Msgctxt GETTEXT_CONTEXT_GLUE Msgid, Msgid, MsgidPlural, N, Category)
-
-static inline const char *
-pgettext_aux (const char *domain,
-	      const char *msg_ctxt_id, const char *msgid,
-	      int category)
-{
-  const char *translation = dcgettext (domain, msg_ctxt_id, category);
-  if (translation == msg_ctxt_id)
-    return msgid;
-  else
-    return translation;
-}
-
-static inline const char *
-npgettext_aux (const char *domain,
-	       const char *msg_ctxt_id, const char *msgid,
-	       const char *msgid_plural, unsigned long int n,
-	       int category)
-{
-  const char *translation =
-    dcngettext (domain, msg_ctxt_id, msgid_plural, n, category);
-  if (translation == msg_ctxt_id || translation == msgid_plural)
-    return (n == 1 ? msgid : msgid_plural);
-  else
-    return translation;
-}
-
-/* The same thing extended for non-constant arguments.  Here MSGCTXT and MSGID
-   can be arbitrary expressions.  But for string literals these macros are
-   less efficient than those above.  */
-
-#include <string.h>
-
-#define _LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS 1
-
-#if !_LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS
-#include <stdlib.h>
-#endif
-
-#define pgettext_expr(Msgctxt, Msgid) \
-  dcpgettext_expr (NULL, Msgctxt, Msgid, LC_MESSAGES)
-#define dpgettext_expr(Domainname, Msgctxt, Msgid) \
-  dcpgettext_expr (Domainname, Msgctxt, Msgid, LC_MESSAGES)
-
-static inline const char *
-dcpgettext_expr (const char *domain,
-		 const char *msgctxt, const char *msgid,
-		 int category)
-{
-  size_t msgctxt_len = strlen (msgctxt) + 1;
-  size_t msgid_len = strlen (msgid) + 1;
-  const char *translation;
-#if _LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS
-  char msg_ctxt_id[msgctxt_len + msgid_len];
-#else
-  char buf[1024];
-  char *msg_ctxt_id =
-    (msgctxt_len + msgid_len <= sizeof (buf)
-     ? buf
-     : (char *) malloc (msgctxt_len + msgid_len));
-  if (msg_ctxt_id != NULL)
-#endif
-    {
-      memcpy (msg_ctxt_id, msgctxt, msgctxt_len - 1);
-      msg_ctxt_id[msgctxt_len - 1] = '\004';
-      memcpy (msg_ctxt_id + msgctxt_len, msgid, msgid_len);
-      translation = dcgettext (domain, msg_ctxt_id, category);
-#if !_LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS
-      if (msg_ctxt_id != buf)
-	free (msg_ctxt_id);
-#endif
-      if (translation != msg_ctxt_id)
-	return translation;
-    }
-  return msgid;
-}
-
-#define npgettext_expr(Msgctxt, Msgid, MsgidPlural, N) \
-  dcnpgettext_expr (NULL, Msgctxt, Msgid, MsgidPlural, N, LC_MESSAGES)
-#define dnpgettext_expr(Domainname, Msgctxt, Msgid, MsgidPlural, N) \
-  dcnpgettext_expr (Domainname, Msgctxt, Msgid, MsgidPlural, N, LC_MESSAGES)
-
-static inline const char *
-dcnpgettext_expr (const char *domain,
-		  const char *msgctxt, const char *msgid,
-		  const char *msgid_plural, unsigned long int n,
-		  int category)
-{
-  size_t msgctxt_len = strlen (msgctxt) + 1;
-  size_t msgid_len = strlen (msgid) + 1;
-  const char *translation;
-#if _LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS
-  char msg_ctxt_id[msgctxt_len + msgid_len];
-#else
-  char buf[1024];
-  char *msg_ctxt_id =
-    (msgctxt_len + msgid_len <= sizeof (buf)
-     ? buf
-     : (char *) malloc (msgctxt_len + msgid_len));
-  if (msg_ctxt_id != NULL)
-#endif
-    {
-      memcpy (msg_ctxt_id, msgctxt, msgctxt_len - 1);
-      msg_ctxt_id[msgctxt_len - 1] = '\004';
-      memcpy (msg_ctxt_id + msgctxt_len, msgid, msgid_len);
-      translation = dcngettext (domain, msg_ctxt_id, msgid_plural, n, category);
-#if !_LIBGETTEXT_HAVE_VARIABLE_SIZE_ARRAYS
-      if (msg_ctxt_id != buf)
-	free (msg_ctxt_id);
-#endif
-      if (!(translation == msg_ctxt_id || translation == msgid_plural))
-	return translation;
-    }
-  return (n == 1 ? msgid : msgid_plural);
-}
-
-#endif /* _LIBGETTEXT_H */

m4/getopt.m4

@@ -1,5 +1,5 @@
-# getopt.m4 serial 13
-dnl Copyright (C) 2002, 2003, 2004, 2005, 2006 Free Software Foundation, Inc.
+# getopt.m4 serial 14 (modified version)
+dnl Copyright (C) 2002-2006, 2008 Free Software Foundation, Inc.
 dnl This file is free software; the Free Software Foundation
 dnl gives unlimited permission to copy and/or distribute it,
 dnl with or without modifications, as long as this notice is preserved.
@@ -13,7 +13,6 @@ AC_DEFUN([gl_GETOPT_SUBSTITUTE],
   AC_LIBOBJ([getopt])
   AC_LIBOBJ([getopt1])
   gl_GETOPT_SUBSTITUTE_HEADER
-  gl_PREREQ_GETOPT
 ])
 
 AC_DEFUN([gl_GETOPT_SUBSTITUTE_HEADER],
@@ -31,41 +30,32 @@ AC_DEFUN([gl_GETOPT_CHECK_HEADERS],
     AC_CHECK_HEADERS([getopt.h], [], [GETOPT_H=getopt.h])
   fi
 
+  dnl BSD getopt_long uses a way to reset option processing, that is different
+  dnl from GNU and Solaris (which copied the GNU behavior). We support both
+  dnl GNU and BSD style resetting of getopt_long(), so there's no need to use
+  dnl GNU getopt_long() on BSD due to different resetting style.
+  dnl
+  dnl With getopt_long(), some BSD versions have a bug in handling optional
+  dnl arguments. This bug appears only if the environment variable
+  dnl POSIXLY_CORRECT has been set, so it shouldn't be too bad in most
+  dnl cases; probably most don't have that variable set. But if we actually
+  dnl hit this bug, it is a real problem due to our heavy use of optional
+  dnl arguments.
+  dnl
+  dnl According to CVS logs, the bug was introduced in OpenBSD in 2003-09-22
+  dnl and copied to FreeBSD in 2004-02-24. It was fixed in both in 2006-09-22,
+  dnl so the affected versions shouldn't be popular anymore anyway. NetBSD
+  dnl never had this bug. TODO: What about Darwin and others?
   if test -z "$GETOPT_H"; then
-    AC_CHECK_FUNCS([getopt_long_only], [], [GETOPT_H=getopt.h])
-  fi
-
-  dnl BSD getopt_long uses an incompatible method to reset option processing,
-  dnl and (as of 2004-10-15) mishandles optional option-arguments.
-  if test -z "$GETOPT_H"; then
-    AC_CHECK_DECL([optreset], [GETOPT_H=getopt.h], [], [#include <getopt.h>])
+    AC_CHECK_DECL([optreset],
+      [AC_DEFINE([HAVE_OPTRESET], 1,
+        [Define to 1 if getopt.h declares extern int optreset.])],
+      [], [#include <getopt.h>])
   fi
 
   dnl Solaris 10 getopt doesn't handle `+' as a leading character in an
-  dnl option string (as of 2005-05-05).
-  if test -z "$GETOPT_H"; then
-    AC_CACHE_CHECK([for working GNU getopt function], [gl_cv_func_gnu_getopt],
-      [AC_RUN_IFELSE(
-	[AC_LANG_PROGRAM([#include <getopt.h>],
-	   [[
-	     char *myargv[3];
-	     myargv[0] = "conftest";
-	     myargv[1] = "-+";
-	     myargv[2] = 0;
-	     return getopt (2, myargv, "+a") != '?';
-	   ]])],
-	[gl_cv_func_gnu_getopt=yes],
-	[gl_cv_func_gnu_getopt=no],
-	[dnl cross compiling - pessimistically guess based on decls
-	 dnl Solaris 10 getopt doesn't handle `+' as a leading character in an
-	 dnl option string (as of 2005-05-05).
-	 AC_CHECK_DECL([getopt_clip],
-	   [gl_cv_func_gnu_getopt=no], [gl_cv_func_gnu_getopt=yes],
-	   [#include <getopt.h>])])])
-    if test "$gl_cv_func_gnu_getopt" = "no"; then
-      GETOPT_H=getopt.h
-    fi
-  fi
+  dnl option string (as of 2005-05-05). We don't use that feature, so this
+  dnl is not a problem for us. Thus, the respective test was removed here.
 ])
 
 AC_DEFUN([gl_GETOPT_IFELSE],
@@ -75,9 +65,3 @@ AC_DEFUN([gl_GETOPT_IFELSE],
 ])
 
 AC_DEFUN([gl_GETOPT], [gl_GETOPT_IFELSE([gl_GETOPT_SUBSTITUTE])])
-
-# Prerequisites of lib/getopt*.
-AC_DEFUN([gl_PREREQ_GETOPT],
-[
-  AC_CHECK_DECLS_ONCE([getenv])
-])

po/LINGUAS

@@ -1 +0,0 @@
-fi

po/POTFILES.in

@@ -1,13 +1,10 @@
 # List of source files which contain translatable strings.
-src/lzma/help.c
-src/lzma/alloc.c
-src/lzma/args.c
-src/lzma/error.c
-src/lzma/io.c
-src/lzma/list.c
-src/lzma/main.c
-src/lzma/options.c
-src/lzma/process.c
-src/lzma/suffix.c
-src/lzma/util.c
-lib/getopt.c
+src/xz/args.c
+src/xz/hardware.c
+src/xz/io.c
+src/xz/main.c
+src/xz/message.c
+src/xz/options.c
+src/xz/process.c
+src/xz/suffix.c
+src/xz/util.c

po/fi.po

@@ -1,446 +0,0 @@
-# translation of fi.po to Finnish
-# This file is put in the public domain.
-#
-msgid ""
-msgstr ""
-"Project-Id-Version: fi\n"
-"Report-Msgid-Bugs-To: lasse.collin@tukaani.org\n"
-"POT-Creation-Date: 2007-12-10 14:34+0200\n"
-"PO-Revision-Date: 2007-08-09 22:14+0300\n"
-"Last-Translator: Lasse Collin <lasse.collin@tukaani.org>\n"
-"Language-Team: Finnish\n"
-"MIME-Version: 1.0\n"
-"Content-Type: text/plain; charset=UTF-8\n"
-"Content-Transfer-Encoding: 8bit\n"
-
-#: src/lzma/help.c:28
-#, c-format
-msgid "Try `%s --help' for more information."
-msgstr "Lisätietoja saa komennolla \"%s --help\"."
-
-#: src/lzma/help.c:36
-#, c-format
-msgid ""
-"Usage: %s [OPTION]... [FILE]...\n"
-"Compress or decompress FILEs in the .lzma format.\n"
-"\n"
-msgstr ""
-"Käyttö: %s [VALITSIN]... [TIEDOSTO]...\n"
-"Pakkaa tai purkaa .lzma-muotoiset TIEDOSTOt\n"
-"\n"
-
-#: src/lzma/help.c:40
-msgid "Mandatory arguments to long options are mandatory for short options too.\n"
-msgstr "Pitkien valitsinten pakolliset argumentit ovat pakollisia myös lyhyille.\n"
-
-#: src/lzma/help.c:44
-msgid ""
-" Operation mode:\n"
-"\n"
-"  -z, --compress      force compression\n"
-"  -d, --decompress    force decompression\n"
-"  -t, --test          test compressed file integrity\n"
-"  -l, --list          list block sizes, total sizes, and possible metadata\n"
-msgstr ""
-
-#: src/lzma/help.c:53
-msgid ""
-" Operation modifiers:\n"
-"\n"
-"  -k, --keep          keep (don't delete) input files\n"
-"  -f, --force         force overwrite of output file and (de)compress links\n"
-"  -c, --stdout        write to standard output and don't delete input files\n"
-"  -S, --suffix=.SUF   use suffix `.SUF' on compressed files instead of `.lzma'\n"
-"  -F, --format=FMT    file format to encode or decode; possible values are\n"
-"                      `auto', `native', `single', `multi', and `alone'\n"
-"      --files=[FILE]  read filenames to process from FILE; if FILE is\n"
-"                      omitted, filenames are read from the standard input;\n"
-"                      filenames must be terminated with the newline character\n"
-"      --files0=[FILE] like --files but use the nul byte as terminator\n"
-msgstr ""
-
-#: src/lzma/help.c:68
-msgid ""
-" Compression presets and basic compression options:\n"
-"\n"
-"  -1 .. -2            fast compression\n"
-"  -3 .. -6            good compression\n"
-"  -7 .. -9            excellent compression, but needs a lot of memory;\n"
-"                      default is -7 if memory limit allows\n"
-"\n"
-"  -C, --check=CHECK   integrity check type: `crc32', `crc64' (default),\n"
-"                      or `sha256'\n"
-msgstr ""
-
-#: src/lzma/help.c:80
-msgid ""
-" Custom filter chain for compression (alternative for using presets):\n"
-"\n"
-"  --lzma=[OPTS]       LZMA filter; OPTS is a comma-separated list of zero or\n"
-"                      more of the following options (valid values; default):\n"
-"                        dict=NUM   dictionary size in bytes (1 - 1Gi; 8Mi)\n"
-"                        lc=NUM     number of literal context bits (0-8; 3)\n"
-"                        lp=NUM     number of literal position bits (0-4; 0)\n"
-"                        pb=NUM     number of position bits (0-4; 2)\n"
-"                        mode=MODE  compression mode (`fast' or `best'; `best')\n"
-"                        fb=NUM     number of fast bytes (5-273; 128)\n"
-"                        mf=NAME    match finder (hc3, hc4, bt2, bt3, bt4; bt4)\n"
-"                        mfc=NUM    match finder cycles; 0=automatic (default)\n"
-"\n"
-"  --x86               x86 filter (sometimes called BCJ filter)\n"
-"  --powerpc           PowerPC (big endian) filter\n"
-"  --ia64              IA64 (Itanium) filter\n"
-"  --arm               ARM filter\n"
-"  --armthumb          ARM-Thumb filter\n"
-"  --sparc             SPARC filter\n"
-"\n"
-"  --copy              No filtering (useful only when specified alone)\n"
-"  --subblock=[OPTS]   Subblock filter; valid OPTS (valid values; default):\n"
-"                        size=NUM    number of bytes of data per subblock\n"
-"                                    (1 - 256Mi; 4Ki)\n"
-"                        rle=NUM     run-length encoder chunk size (0-256; 0)\n"
-msgstr ""
-
-#: src/lzma/help.c:120
-msgid ""
-" Resource usage options:\n"
-"\n"
-"  -M, --memory=NUM    use roughly NUM bytes of memory at maximum\n"
-"  -T, --threads=NUM   use at maximum of NUM (de)compression threads\n"
-msgstr ""
-
-#: src/lzma/help.c:129
-msgid ""
-" Other options:\n"
-"\n"
-"  -q, --quiet         suppress warnings; specify twice to suppress errors too\n"
-"  -v, --verbose       be verbose; specify twice for even more verbose\n"
-"\n"
-"  -h, --help          display this help and exit\n"
-"  -V, --version       display version and license information and exit\n"
-msgstr ""
-
-#: src/lzma/help.c:137
-msgid "With no FILE, or when FILE is -, read standard input.\n"
-msgstr "Jos TIEDOSTOa ei ole annettu, tai se on \"-\", luetaan vakiosyötettä.\n"
-
-#: src/lzma/help.c:143
-msgid "On this system and configuration, the tool will use"
-msgstr "Tässä järjestelmässä näillä asetuksilla, tämä työkalu käyttää"
-
-#: src/lzma/help.c:144
-#, c-format
-msgid "  * roughly %zu MiB of memory at maximum; and\n"
-msgstr "  * korkeintaan %zu MiB keskusmuistia (suurpiirteinen rajoitus); ja\n"
-
-#: src/lzma/help.c:147
-msgid ""
-"  * at maximum of one thread for (de)compression.\n"
-"\n"
-msgstr ""
-"  * korkeintaan yhtä säiettä pakkaukseen tai purkuun.\n"
-"\n"
-
-#: src/lzma/help.c:151
-#, c-format
-msgid "Report bugs to <%s> (in English or Finnish).\n"
-msgstr ""
-"Lähetä raportit ohjelmistovioista (englanniksi tai suomeksi)\n"
-"osoitteeseen <%s>.\n"
-
-#: src/lzma/args.c:128
-msgid "Maximum number of filters is seven"
-msgstr "Suotimia voi olla korkeintaan seitsemän"
-
-#: src/lzma/args.c:193
-#, c-format
-msgid "%s: Invalid filename suffix"
-msgstr "%s: Virheellinen tiedostonimen pääte"
-
-#: src/lzma/args.c:327
-#, c-format
-msgid "%s: Unknown file format type"
-msgstr "%s: Tuntematon tiedostomuodon tyyppi"
-
-#: src/lzma/args.c:353
-#, c-format
-msgid "%s: Unknown integrity check type"
-msgstr "%s: Tuntematon eheystarkisteen tyyppi"
-
-#: src/lzma/args.c:371
-msgid "Only one file can be specified with `--files'or `--files0'."
-msgstr ""
-
-#: src/lzma/args.c:421
-msgid "The environment variable LZMA_OPT contains too many arguments"
-msgstr "Ympäristömuuttuja LZMA_OPT sisältää liian monta argumenttia"
-
-#: src/lzma/args.c:485
-msgid "Memory usage limit is too small for any internal filter preset"
-msgstr ""
-
-#: src/lzma/args.c:500
-msgid "Memory usage limit is too small for the given filter setup"
-msgstr "Muistinkäyttörajoitus on liian pieni annetuille suodinasetusille"
-
-#: src/lzma/error.c:35
-msgid "Operation successful"
-msgstr ""
-
-#: src/lzma/error.c:38
-msgid "Operation finished successfully"
-msgstr ""
-
-#: src/lzma/error.c:41 src/lzma/error.c:153
-msgid "Internal error (bug)"
-msgstr "Sisäinen virhe (bugi)"
-
-#: src/lzma/error.c:44
-msgid "Compressed data is corrupt"
-msgstr "Pakattu data on korruptoitunut"
-
-#: src/lzma/error.c:50
-msgid "Unexpected end of input"
-msgstr ""
-
-#: src/lzma/error.c:53
-msgid "Unsupported options"
-msgstr ""
-
-#: src/lzma/error.c:56
-msgid "Unsupported integrity check type"
-msgstr ""
-
-#: src/lzma/error.c:87 src/lzma/error.c:93
-#, c-format
-msgid "Writing to standard output failed: %s"
-msgstr ""
-
-#: src/lzma/io.c:81
-#, c-format
-msgid "Cannot get file descriptor of the current directory: %s"
-msgstr ""
-
-#: src/lzma/io.c:123 src/lzma/util.c:127
-#, c-format
-msgid "%s: Invalid filename"
-msgstr "%s: Virheellinen tiedostonimi"
-
-#: src/lzma/io.c:130 src/lzma/io.c:233
-#, c-format
-msgid "Cannot change directory: %s"
-msgstr "Hakemiston vaihtaminen epäonnistui: %s"
-
-#: src/lzma/io.c:135
-#, c-format
-msgid "%s: File seems to be moved, not removing"
-msgstr "%s: Tiedosto näyttää olevan siirretty, ei poisteta"
-
-#: src/lzma/io.c:141
-#, c-format
-msgid "%s: Cannot remove: %s"
-msgstr ""
-
-#: src/lzma/io.c:164
-#, c-format
-msgid "%s: Cannot set the file owner: %s"
-msgstr "%s: Tiedoston omistajuuden asetus epäonnistui: %s"
-
-#: src/lzma/io.c:170
-#, c-format
-msgid "%s: Cannot set the file group: %s"
-msgstr "%s: Tiedoston ryhmän asetus epäonnistui: %s"
-
-#: src/lzma/io.c:189
-#, c-format
-msgid "%s: Cannot set the file permissions: %s"
-msgstr "%s: Tiedoston oikeuksien asetus epäonnistui: %s"
-
-#: src/lzma/io.c:252
-#, c-format
-msgid "%s: Cannot open the directory containing the file: %s"
-msgstr ""
-
-#: src/lzma/io.c:313
-#, c-format
-msgid "%s: Is a symbolic link, skipping"
-msgstr "%s: Symbolinen linkki, ohitetaan"
-
-#: src/lzma/io.c:330
-#, c-format
-msgid "%s: Is a directory, skipping"
-msgstr "%s: Hakemisto, ohitetaan"
-
-#: src/lzma/io.c:337 src/lzma/list.c:445
-#, c-format
-msgid "%s: Not a regular file, skipping"
-msgstr "%s: Ei ole tavallinen tiedosto, ohitetaan"
-
-#: src/lzma/io.c:348
-#, c-format
-msgid "%s: File has setuid or setgid bit set, skipping"
-msgstr "%s: Tiedostolla on setuid- tai setgid-bitti asetettuna, ohitetaan"
-
-#: src/lzma/io.c:355
-#, c-format
-msgid "%s: File has sticky bit set, skipping"
-msgstr "%s: Tiedostolla on sticky-bitti asetettuna, ohitetaan"
-
-#: src/lzma/io.c:362
-#, c-format
-msgid "%s: Input file has more than one hard link, skipping"
-msgstr "%s: Tiedostoon viittaa useampi kuin yksi linkki, ohitetaan"
-
-#: src/lzma/io.c:390 src/lzma/io.c:468
-#, c-format
-msgid "%s: Closing the file failed: %s"
-msgstr "%s: Tiedoston sulkeminen epäonnistui: %s"
-
-#: src/lzma/io.c:611
-#, c-format
-msgid "%s: Read error: %s"
-msgstr "%s: Lukeminen epäonnistui: %s"
-
-#: src/lzma/io.c:654
-#, c-format
-msgid "%s: Write error: %s"
-msgstr "%s: Kirjoitus epäonnistui: %s"
-
-#: src/lzma/list.c:451
-#, c-format
-msgid "%s: File is empty"
-msgstr ""
-
-#: src/lzma/main.c:57
-msgid "Cannot establish signal handlers"
-msgstr "Signaalikäsittelijöiden asetus epäonnistui"
-
-#: src/lzma/main.c:75
-msgid "Compressed data not read from a terminal."
-msgstr "Pakattua dataa ei lueta päätteeltä."
-
-#: src/lzma/main.c:77 src/lzma/main.c:92
-msgid "Use `--force' to force decompression."
-msgstr "Käytä valitsinta \"--force\" pakottaaksesi purun."
-
-#: src/lzma/main.c:90
-msgid "Compressed data not written to a terminal."
-msgstr "Pakattua dataa ei kirjoiteta päätteelle."
-
-#: src/lzma/main.c:117
-#, fuzzy, c-format
-msgid "%s: Error reading filenames: %s"
-msgstr "%s: Tiedoston sulkeminen epäonnistui: %s"
-
-#: src/lzma/main.c:122
-#, c-format
-msgid "%s: Unexpected end of input when reading filenames"
-msgstr ""
-
-#: src/lzma/main.c:212
-msgid "Cannot read data from standard input when reading filenames from standard input"
-msgstr "Dataa ei voida lukea oletussyötteestä, kun tiedostonimiä luetaan oletussyötteestä"
-
-#: src/lzma/options.c:83
-#, c-format
-msgid "%s: Options must be `name=value' pairs separated with commas"
-msgstr "%s: Asetusten tulee olla \"nimi=arvo\"-pareja, jotka on eroteltu pilkuilla"
-
-#: src/lzma/options.c:111
-#, c-format
-msgid "%s: Invalid option value"
-msgstr "%s: Virheellinen asetuksen arvo"
-
-#: src/lzma/options.c:124
-#, c-format
-msgid "%s: Invalid option name"
-msgstr "%s: Virheellinen asetuksen nimi"
-
-#: src/lzma/process.c:142
-#, c-format
-msgid "Cannot create a thread: %s"
-msgstr "Säikeen luonti epäonnistui: %s"
-
-#: src/lzma/suffix.c:85
-#, c-format
-msgid "%s: Filename has an unknown suffix, skipping"
-msgstr "%s: Tiedostonimellä on tuntematon pääte, ohitetaan"
-
-#: src/lzma/suffix.c:114
-#, c-format
-msgid "%s: File already has `%s' suffix, skipping"
-msgstr "%s: Tiedostolla on jo \"%s\"-pääte, ohitetaan"
-
-#: src/lzma/util.c:44
-#, c-format
-msgid "%s: Value is not a non-negative decimal integer"
-msgstr ""
-
-#: src/lzma/util.c:84
-#, c-format
-msgid "%s: Invalid multiplier suffix. Valid suffixes:"
-msgstr ""
-
-#: src/lzma/util.c:105
-#, c-format
-msgid "Value of the option `%s' must be in the range [%llu, %llu]"
-msgstr "Asetuksen \"%s\" arvon tulee olla välillä [%llu, %llu]"
-
-#: src/lzma/util.c:177
-msgid "Empty filename, skipping"
-msgstr "Tyhjä tiedostonimi, ohitetaan"
-
-#: lib/getopt.c:531 lib/getopt.c:547
-#, c-format
-msgid "%s: option `%s' is ambiguous\n"
-msgstr ""
-
-#: lib/getopt.c:580 lib/getopt.c:584
-#, c-format
-msgid "%s: option `--%s' doesn't allow an argument\n"
-msgstr ""
-
-#: lib/getopt.c:593 lib/getopt.c:598
-#, c-format
-msgid "%s: option `%c%s' doesn't allow an argument\n"
-msgstr ""
-
-#: lib/getopt.c:641 lib/getopt.c:660 lib/getopt.c:976 lib/getopt.c:995
-#, c-format
-msgid "%s: option `%s' requires an argument\n"
-msgstr ""
-
-#: lib/getopt.c:698 lib/getopt.c:701
-#, c-format
-msgid "%s: unrecognized option `--%s'\n"
-msgstr ""
-
-#: lib/getopt.c:709 lib/getopt.c:712
-#, c-format
-msgid "%s: unrecognized option `%c%s'\n"
-msgstr ""
-
-#: lib/getopt.c:764 lib/getopt.c:767
-#, c-format
-msgid "%s: illegal option -- %c\n"
-msgstr ""
-
-#: lib/getopt.c:773 lib/getopt.c:776
-#, c-format
-msgid "%s: invalid option -- %c\n"
-msgstr "%s: virheellinen valitsin -- %c\n"
-
-#: lib/getopt.c:828 lib/getopt.c:844 lib/getopt.c:1048 lib/getopt.c:1066
-#, c-format
-msgid "%s: option requires an argument -- %c\n"
-msgstr ""
-
-#: lib/getopt.c:897 lib/getopt.c:913
-#, c-format
-msgid "%s: option `-W %s' is ambiguous\n"
-msgstr ""
-
-#: lib/getopt.c:937 lib/getopt.c:955
-#, c-format
-msgid "%s: option `-W %s' doesn't allow an argument\n"
-msgstr ""

src/Makefile.am

@@ -12,5 +12,5 @@
 ##  Lesser General Public License for more details.
 ##
 
-SUBDIRS = liblzma lzma lzmadec scripts
+SUBDIRS = liblzma xz xzdec scripts
 EXTRA_DIST = common

src/common/bswap.h

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
-/// \file       check_byteswap.h
-/// \brief      Byteswapping needed by the checks
+/// \file       bswap.h
+/// \brief      Byte swapping
 //
 //  This code has been put into the public domain.
 //
@@ -11,24 +11,34 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#ifndef LZMA_CHECK_BYTESWAP_H
-#define LZMA_CHECK_BYTESWAP_H
+#ifndef LZMA_BSWAP_H
+#define LZMA_BSWAP_H
 
-#ifdef HAVE_CONFIG_H
-#	include <config.h>
-#endif
+// NOTE: We assume that config.h is already #included.
+
+// At least glibc has byteswap.h which contains inline assembly code for
+// byteswapping. Some systems have byteswap.h but lack one or more of the
+// bswap_xx macros/functions, which is why we check them separately even
+// if byteswap.h is available.
 
-// byteswap.h is a GNU extension. It contains inline assembly versions
-// for byteswapping. When byteswap.h is not available, we use generic code.
 #ifdef HAVE_BYTESWAP_H
 #	include <byteswap.h>
-#else
+#endif
+
+#ifndef HAVE_BSWAP_16
+#	define bswap_16(num) \
+		(((num) << 8) | ((num) >> 8))
+#endif
+
+#ifndef HAVE_BSWAP_32
 #	define bswap_32(num) \
 		( (((num) << 24)                       ) \
 		| (((num) <<  8) & UINT32_C(0x00FF0000)) \
 		| (((num) >>  8) & UINT32_C(0x0000FF00)) \
 		| (((num) >> 24)                       ) )
+#endif
 
+#ifndef HAVE_BSWAP_64
 #	define bswap_64(num) \
 		( (((num) << 56)                               ) \
 		| (((num) << 40) & UINT64_C(0x00FF000000000000)) \

src/common/integer.h

@@ -0,0 +1,171 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       integer.h
+/// \brief      Reading and writing integers from and to buffers
+//
+//  This code has been put into the public domain.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#ifndef LZMA_INTEGER_H
+#define LZMA_INTEGER_H
+
+// On big endian, we need byte swapping. These macros may be used outside
+// this file, so don't put these inside HAVE_FAST_UNALIGNED_ACCESS.
+#ifdef WORDS_BIGENDIAN
+#	include "bswap.h"
+#	define integer_le_16(n) bswap_16(n)
+#	define integer_le_32(n) bswap_32(n)
+#	define integer_le_64(n) bswap_64(n)
+#else
+#	define integer_le_16(n) (n)
+#	define integer_le_32(n) (n)
+#	define integer_le_64(n) (n)
+#endif
+
+
+// I'm aware of AC_CHECK_ALIGNED_ACCESS_REQUIRED from Autoconf archive, but
+// it's not useful here. We don't care if unaligned access is supported,
+// we care if it is fast. Some systems can emulate unaligned access in
+// software, which is horribly slow; we want to use byte-by-byte access on
+// such systems but the Autoconf test would detect such a system as
+// supporting unaligned access.
+//
+// NOTE: HAVE_FAST_UNALIGNED_ACCESS indicates only support for 16-bit and
+// 32-bit integer loads and stores. 64-bit integers may or may not work.
+// That's why 64-bit functions are commented out.
+//
+// TODO: Big endian PowerPC supports byte swapping load and store instructions
+// that also allow unaligned access. Inline assembler could be OK for that.
+//
+// Performance of these functions isn't that important until LZMA3, but it
+// doesn't hurt to have these ready already.
+#ifdef HAVE_FAST_UNALIGNED_ACCESS
+
+static inline uint16_t
+integer_read_16(const uint8_t buf[static 2])
+{
+	uint16_t ret = *(const uint16_t *)(buf);
+	return integer_le_16(ret);
+}
+
+
+static inline uint32_t
+integer_read_32(const uint8_t buf[static 4])
+{
+	uint32_t ret = *(const uint32_t *)(buf);
+	return integer_le_32(ret);
+}
+
+
+/*
+static inline uint64_t
+integer_read_64(const uint8_t buf[static 8])
+{
+	uint64_t ret = *(const uint64_t *)(buf);
+	return integer_le_64(ret);
+}
+*/
+
+
+static inline void
+integer_write_16(uint8_t buf[static 2], uint16_t num)
+{
+	*(uint16_t *)(buf) = integer_le_16(num);
+}
+
+
+static inline void
+integer_write_32(uint8_t buf[static 4], uint32_t num)
+{
+	*(uint32_t *)(buf) = integer_le_32(num);
+}
+
+
+/*
+static inline void
+integer_write_64(uint8_t buf[static 8], uint64_t num)
+{
+	*(uint64_t *)(buf) = integer_le_64(num);
+}
+*/
+
+
+#else
+
+static inline uint16_t
+integer_read_16(const uint8_t buf[static 2])
+{
+	uint16_t ret = buf[0] | (buf[1] << 8);
+	return ret;
+}
+
+
+static inline uint32_t
+integer_read_32(const uint8_t buf[static 4])
+{
+	uint32_t ret = buf[0];
+	ret |= (uint32_t)(buf[1]) << 8;
+	ret |= (uint32_t)(buf[2]) << 16;
+	ret |= (uint32_t)(buf[3]) << 24;
+	return ret;
+}
+
+
+/*
+static inline uint64_t
+integer_read_64(const uint8_t buf[static 8])
+{
+	uint64_t ret = buf[0];
+	ret |= (uint64_t)(buf[1]) << 8;
+	ret |= (uint64_t)(buf[2]) << 16;
+	ret |= (uint64_t)(buf[3]) << 24;
+	ret |= (uint64_t)(buf[4]) << 32;
+	ret |= (uint64_t)(buf[5]) << 40;
+	ret |= (uint64_t)(buf[6]) << 48;
+	ret |= (uint64_t)(buf[7]) << 56;
+	return ret;
+}
+*/
+
+
+static inline void
+integer_write_16(uint8_t buf[static 2], uint16_t num)
+{
+	buf[0] = (uint8_t)(num);
+	buf[1] = (uint8_t)(num >> 8);
+}
+
+
+static inline void
+integer_write_32(uint8_t buf[static 4], uint32_t num)
+{
+	buf[0] = (uint8_t)(num);
+	buf[1] = (uint8_t)(num >> 8);
+	buf[2] = (uint8_t)(num >> 16);
+	buf[3] = (uint8_t)(num >> 24);
+}
+
+
+/*
+static inline void
+integer_write_64(uint8_t buf[static 8], uint64_t num)
+{
+	buf[0] = (uint8_t)(num);
+	buf[1] = (uint8_t)(num >> 8);
+	buf[2] = (uint8_t)(num >> 16);
+	buf[3] = (uint8_t)(num >> 24);
+	buf[4] = (uint8_t)(num >> 32);
+	buf[5] = (uint8_t)(num >> 40);
+	buf[6] = (uint8_t)(num >> 48);
+	buf[7] = (uint8_t)(num >> 56);
+}
+*/
+
+#endif
+
+#endif

src/common/mythread.h

@@ -0,0 +1,34 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       mythread.h
+/// \brief      Wrappers for threads
+//
+//  Author: Lasse Collin
+//  This file has been put into the public domain.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "sysdefs.h"
+
+
+#ifdef HAVE_PTHREAD
+#	include <pthread.h>
+
+#	define mythread_once(func) \
+	do { \
+		static pthread_once_t once_ = PTHREAD_ONCE_INIT; \
+		pthread_once(&once_, &func); \
+	} while (0)
+
+#else
+
+#	define mythread_once(func) \
+	do { \
+		static bool once_ = false; \
+		if (!once_) { \
+			func(); \
+			once_ = true; \
+		} \
+	} while (0)
+
+#endif

src/common/physmem.h

@@ -14,17 +14,6 @@
 #ifndef PHYSMEM_H
 #define PHYSMEM_H
 
-#ifdef HAVE_CONFIG_H
-#	include <config.h>
-#endif
-
-#include <sys/types.h>
-#include <inttypes.h>
-
-#ifdef HAVE_LIMITS_H
-#	include <limits.h>
-#endif
-
 #if defined(HAVE_PHYSMEM_SYSCTL) || defined(HAVE_NCPU_SYSCTL)
 #	ifdef HAVE_SYS_PARAM_H
 #		include <sys/param.h>
@@ -34,6 +23,10 @@
 #	endif
 #endif
 
+#if defined(HAVE_PHYSMEM_SYSCONF) || defined(HAVE_NCPU_SYSCONF)
+#	include <unistd.h>
+#endif
+
 
 /// \brief      Get the amount of physical memory in bytes
 ///

src/common/sysdefs.h

@@ -31,10 +31,98 @@
 #	include <config.h>
 #endif
 
-#include "lzma.h"
+// size_t and NULL
+#include <stddef.h>
+
+#ifdef HAVE_INTTYPES_H
+#	include <inttypes.h>
+#endif
+
+// C99 says that inttypes.h always includes stdint.h, but some systems
+// don't do that, and require including stdint.h separately.
+#ifdef HAVE_STDINT_H
+#	include <stdint.h>
+#endif
+
+// Some pre-C99 systems have SIZE_MAX in limits.h instead of stdint.h. The
+// limits are also used to figure out some macros missing from pre-C99 systems.
+#ifdef HAVE_LIMITS_H
+#	include <limits.h>
+#endif
+
+// Be more compatible with systems that have non-conforming inttypes.h.
+// We assume that int is 32-bit and that long is either 32-bit or 64-bit.
+// Full Autoconf test could be more correct, but this should work well enough.
+// Note that this duplicates some code from lzma.h, but this is better since
+// we can work without inttypes.h thanks to Autoconf tests.
+#ifndef UINT32_C
+#	if UINT_MAX != 4294967295U
+#		error UINT32_C is not defined and unsiged int is not 32-bit.
+#	endif
+#	define UINT32_C(n) n ## U
+#endif
+#ifndef UINT32_MAX
+#	define UINT32_MAX UINT32_C(4294967295)
+#endif
+#ifndef PRIu32
+#	define PRIu32 "u"
+#endif
+#ifndef PRIX32
+#	define PRIX32 "X"
+#endif
+
+#if ULONG_MAX == 4294967295UL
+#	ifndef UINT64_C
+#		define UINT64_C(n) n ## ULL
+#	endif
+#	ifndef PRIu64
+#		define PRIu64 "llu"
+#	endif
+#	ifndef PRIX64
+#		define PRIX64 "llX"
+#	endif
+#else
+#	ifndef UINT64_C
+#		define UINT64_C(n) n ## UL
+#	endif
+#	ifndef PRIu64
+#		define PRIu64 "lu"
+#	endif
+#	ifndef PRIX64
+#		define PRIX64 "lX"
+#	endif
+#endif
+#ifndef UINT64_MAX
+#	define UINT64_MAX UINT64_C(18446744073709551615)
+#endif
+
+// The code currently assumes that size_t is either 32-bit or 64-bit.
+#ifndef SIZE_MAX
+#	if SIZEOF_SIZE_T == 4
+#		define SIZE_MAX UINT32_MAX
+#	elif SIZEOF_SIZE_T == 8
+#		define SIZE_MAX UINT64_MAX
+#	else
+#		error sizeof(size_t) is not 32-bit or 64-bit
+#	endif
+#endif
+#if SIZE_MAX != UINT32_MAX && SIZE_MAX != UINT64_MAX
+#	error sizeof(size_t) is not 32-bit or 64-bit
+#endif
 
 #include <stdlib.h>
+#include <assert.h>
 
+// Pre-C99 systems lack stdbool.h. All the code in LZMA Utils must be written
+// so that it works with fake bool type, for example:
+//
+//    bool foo = (flags & 0x100) != 0;
+//    bool bar = !!(flags & 0x100);
+//
+// This works with the real C99 bool but breaks with fake bool:
+//
+//    bool baz = (flags & 0x100);
+//
 #ifdef HAVE_STDBOOL_H
 #	include <stdbool.h>
 #else
@@ -47,17 +135,8 @@ typedef unsigned char _Bool;
 #	define __bool_true_false_are_defined 1
 #endif
 
-#ifdef HAVE_ASSERT_H
-#	include <assert.h>
-#else
-#	ifdef NDEBUG
-#		define assert(x)
-#	else
-		// TODO: Pretty bad assert() macro.
-#		define assert(x) (!(x) && abort())
-#	endif
-#endif
-
+// string.h should be enough but let's include strings.h and memory.h too if
+// they exists, since that shouldn't do any harm, but may improve portability.
 #ifdef HAVE_STRING_H
 #	include <string.h>
 #endif
@@ -70,24 +149,15 @@ typedef unsigned char _Bool;
 #	include <memory.h>
 #endif
 
+#include "lzma.h"
+
 
 ////////////
 // Macros //
 ////////////
 
-#ifndef HAVE_MEMCPY
-#	define memcpy(dest, src, n) bcopy(src, dest, n)
-#endif
-
-#ifndef HAVE_MEMMOVE
-#	define memmove(dest, src, n) bcopy(src, dest, n)
-#endif
-
-#ifdef HAVE_MEMSET
-#	define memzero(s, n) memset(s, 0, n)
-#else
-#	define memzero(s, n) bzero(s, n)
-#endif
+#undef memzero
+#define memzero(s, n) memset(s, 0, n)
 
 #ifndef MIN
 #	define MIN(x, y) ((x) < (y) ? (x) : (y))
@@ -97,4 +167,8 @@ typedef unsigned char _Bool;
 #	define MAX(x, y) ((x) > (y) ? (x) : (y))
 #endif
 
+#ifndef ARRAY_SIZE
+#	define ARRAY_SIZE(array) (sizeof(array) / sizeof((array)[0]))
+#endif
+
 #endif

src/liblzma/Makefile.am

@@ -22,11 +22,15 @@ liblzma_la_LIBADD = \
 	common/libcommon.la \
 	check/libcheck.la
 
-if COND_FILTER_LZMA
-SUBDIRS += lz lzma rangecoder
+if COND_FILTER_LZ
+SUBDIRS += lz
+liblzma_la_LIBADD += lz/liblz.la
+endif
+
+if COND_FILTER_LZMA1
+SUBDIRS += lzma rangecoder
 liblzma_la_LIBADD += \
-	lz/liblz.la \
-	lzma/liblzma4.la \
+	lzma/liblzma2.la \
 	rangecoder/librangecoder.la
 endif
 
@@ -35,7 +39,12 @@ SUBDIRS += subblock
 liblzma_la_LIBADD += subblock/libsubblock.la
 endif
 
-if COND_MAIN_SIMPLE
+if COND_FILTER_DELTA
+SUBDIRS += delta
+liblzma_la_LIBADD += delta/libdelta.la
+endif
+
+if COND_FILTER_SIMPLE
 SUBDIRS += simple
 liblzma_la_LIBADD += simple/libsimple.la
 endif
@@ -43,5 +52,5 @@ endif
 
 ## pkg-config
 pkgconfigdir = $(libdir)/pkgconfig
-pkgconfig_DATA = lzma.pc
-EXTRA_DIST = lzma.pc.in
+pkgconfig_DATA = liblzma.pc
+EXTRA_DIST = liblzma.pc.in

src/liblzma/api/Makefile.am

@@ -14,25 +14,16 @@
 
 nobase_include_HEADERS = \
 	lzma.h \
-	lzma/alignment.h \
-	lzma/alone.h \
-	lzma/auto.h \
 	lzma/base.h \
+	lzma/bcj.h \
 	lzma/block.h \
 	lzma/check.h \
-	lzma/copy.h \
+	lzma/container.h \
 	lzma/delta.h \
-	lzma/extra.h \
 	lzma/filter.h \
 	lzma/index.h \
-	lzma/info.h \
-	lzma/init.h \
+	lzma/index_hash.h \
 	lzma/lzma.h \
-	lzma/memlimit.h \
-	lzma/metadata.h \
-	lzma/raw.h \
-	lzma/simple.h \
-	lzma/stream.h \
 	lzma/stream_flags.h \
 	lzma/subblock.h \
 	lzma/version.h \

src/liblzma/api/lzma.h

@@ -22,46 +22,165 @@
 #ifndef LZMA_H
 #define LZMA_H
 
+/*****************************
+ * Required standard headers *
+ *****************************/
+
+/**
+ * liblzma API headers need some standard types and macros. To allow
+ * including lzma.h without requiring the application to include other
+ * headers first, lzma.h includes the required standard headers unless
+ * they already seem to be included already or if LZMA_MANUAL_HEADERS
+ * has been defined.
+ *
+ * Here's what types and macros are needed and from which headers:
+ *  - stddef.h: size_t, NULL
+ *  - stdint.h: uint8_t, uint32_t, uint64_t, UINT32_C(n), uint64_C(n),
+ *    UINT32_MAX, UINT64_MAX
+ *
+ * However, inttypes.h is a little more portable than stdint.h, although
+ * inttypes.h declares some unneeded things compared to plain stdint.h.
+ *
+ * The hacks below aren't perfect, specifically they assume that inttypes.h
+ * exists and that it typedefs at least uint8_t, uint32_t, and uint64_t,
+ * and that, in case of incomplete inttypes.h, unsigned int is 32-bit.
+ * If the application already takes care of setting up all the types and
+ * macros properly (for example by using gnulib's stdint.h or inttypes.h),
+ * we try to detect that the macros are already defined and don't include
+ * inttypes.h here again. However, you may define LZMA_MANUAL_HEADERS to
+ * force this file to never include any system headers.
+ *
+ * Some could argue that liblzma API should provide all the required types,
+ * for example lzma_uint64, LZMA_UINT64_C(n), and LZMA_UINT64_MAX. This was
+ * seen unnecessary mess, since most systems already provide all the necessary
+ * types and macros in the standard headers.
+ *
+ * Note that liblzma API still has lzma_bool, because using stdbool.h would
+ * break C89 and C++ programs on many systems. sizeof(bool) in C99 isn't
+ * necessarily the same as sizeof(bool) in C++.
+ */
+
+#ifndef LZMA_MANUAL_HEADERS
+	/*
+	 * I suppose this works portably also in C++. Note that in C++,
+	 * we need to get size_t into the global namespace.
+	 */
+	#include <stddef.h>
+
+	/*
+	 * Skip inttypes.h if we already have all the required macros. If we
+	 * have the macros, we assume that we have the matching typedefs too.
+	 */
+#	if !defined(UINT32_C) || !defined(UINT64_C) \
+			|| !defined(UINT32_MAX) || !defined(UINT64_MAX)
+#		ifdef __cplusplus
+			/*
+			 * C99 sections 7.18.2 and 7.18.4 specify that in C++
+			 * implementations define the limit and constant
+			 * macros only if specifically requested. Note that
+			 * if you want the format macros (PRIu64 etc.) too,
+			 * you need to define __STDC_FORMAT_MACROS before
+			 * including lzma.h, since re-including inttypes.h
+			 * with __STDC_FORMAT_MACROS defined doesn't
+			 * necessarily work.
+			 */
+#			ifndef __STDC_LIMIT_MACROS
+#				define __STDC_LIMIT_MACROS 1
+#			endif
+#			ifndef __STDC_CONSTANT_MACROS
+#				define __STDC_CONSTANT_MACROS 1
+#			endif
+#		endif
+
+#		include <inttypes.h>
+
+		/*
+		 * Some old systems have only the typedefs in inttypes.h, and
+		 * lack all the macros. For those systems, we need a few more
+		 * hacks. We assume that unsigned int is 32-bit and unsigned
+		 * long is either 32-bit or 64-bit. If these hacks aren't
+		 * enough, the application has to setup the types manually
+		 * before including lzma.h.
+		 */
+#		ifndef UINT32_C
+#			define UINT32_C(n) n # U
+#		endif
+
+#		ifndef UINT64_C
+			/* Get ULONG_MAX. */
+#			include <limits.h>
+#			if ULONG_MAX == 4294967295UL
+#				define UINT64_C(n) n ## ULL
+#			else
+#				define UINT64_C(n) n ## UL
+#			endif
+#		endif
+
+#		ifndef UINT32_MAX
+#			define UINT32_MAX (UINT32_C(4294967295))
+#		endif
+
+#		ifndef UINT64_MAX
+#			define UINT64_MAX (UINT64_C(18446744073709551615))
+#		endif
+#	endif
+#endif /* ifdef LZMA_MANUAL_HEADERS */
+
+
 /********************
- * External headers *
+ * GNU C extensions *
  ********************/
 
-/* size_t */
-#include <sys/types.h>
-
-/* NULL */
-#include <stddef.h>
-
-/* uint8_t, uint32_t, uint64_t, UINT32_C, UINT64_C, UINT64_MAX. */
-#include <inttypes.h>
-
-
-/******************
- * GCC extensions *
- ******************/
-
 /*
- * GCC extensions are used conditionally in the public API. It doesn't
+ * GNU C extensions are used conditionally in the public API. It doesn't
  * break anything if these are sometimes enabled and sometimes not, only
  * affects warnings and optimizations.
  */
-#if defined(__GNUC__) && __GNUC__ >= 3
+#if __GNUC__ >= 3
 #	ifndef lzma_attribute
 #		define lzma_attribute(attr) __attribute__(attr)
 #	endif
+
 #	ifndef lzma_restrict
 #		define lzma_restrict __restrict__
 #	endif
+
+	/* warn_unused_result was added in GCC 3.4. */
+#	ifndef lzma_attr_warn_unused_result
+#		if __GNUC__ == 3 && __GNUC_MINOR__ < 4
+#			define lzma_attr_warn_unused_result
+#		endif
+#	endif
+
 #else
 #	ifndef lzma_attribute
 #		define lzma_attribute(attr)
 #	endif
+
 #	ifndef lzma_restrict
-#		define lzma_restrict
+#		if __STDC_VERSION__ >= 199901L
+#			define lzma_restrict restrict
+#		else
+#			define lzma_restrict
+#		endif
 #	endif
 #endif
 
 
+#ifndef lzma_attr_pure
+#	define lzma_attr_pure lzma_attribute((__pure__))
+#endif
+
+#ifndef lzma_attr_const
+#	define lzma_attr_const lzma_attribute((__const__))
+#endif
+
+#ifndef lzma_attr_warn_unused_result
+#	define lzma_attr_warn_unused_result \
+		lzma_attribute((__warn_unused_result__))
+#endif
+
+
 /**************
  * Subheaders *
  **************/
@@ -77,37 +196,26 @@ extern "C" {
 #define LZMA_H_INTERNAL 1
 
 /* Basic features */
-#include "lzma/init.h"
+#include "lzma/version.h"
 #include "lzma/base.h"
 #include "lzma/vli.h"
-#include "lzma/filter.h"
 #include "lzma/check.h"
 
 /* Filters */
-#include "lzma/copy.h"
+#include "lzma/filter.h"
 #include "lzma/subblock.h"
-#include "lzma/simple.h"
+#include "lzma/bcj.h"
 #include "lzma/delta.h"
 #include "lzma/lzma.h"
 
-/* Container formats and Metadata */
-#include "lzma/block.h"
-#include "lzma/index.h"
-#include "lzma/extra.h"
-#include "lzma/metadata.h"
-#include "lzma/stream.h"
-#include "lzma/alone.h"
-#include "lzma/raw.h"
-#include "lzma/auto.h"
+/* Container formats */
+#include "lzma/container.h"
 
 /* Advanced features */
-#include "lzma/info.h"
-#include "lzma/alignment.h"
 #include "lzma/stream_flags.h"
-#include "lzma/memlimit.h"
-
-/* Version number */
-#include "lzma/version.h"
+#include "lzma/block.h"
+#include "lzma/index.h"
+#include "lzma/index_hash.h"
 
 /*
  * All subheaders included. Undefine LZMA_H_INTERNAL to prevent applications

src/liblzma/api/lzma/alignment.h

@@ -1,60 +0,0 @@
-/**
- * \file        lzma/alignment.h
- * \brief       Calculating input and output alignment of filter chains
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Calculates the preferred alignment of the input data
- *
- * FIXME desc
- */
-extern uint32_t lzma_alignment_input(
-		const lzma_options_filter *filters, uint32_t guess);
-
-
-/**
- * \brief       Calculates the alignment of the encoded output
- *
- * Knowing the alignment of the output data is useful e.g. in the Block
- * encoder which tries to align the Compressed Data field optimally.
- *
- * \param       filters   Pointer to lzma_options_filter array, whose last
- *                        member must have .id = LZMA_VLI_VALUE_UNKNOWN.
- * \param       guess     The value to return if the alignment of the output
- *                        is the same as the alignment of the input data.
- *                        If you want to always detect this special case,
- *                        this guess to zero; this function never returns
- *                        zero unless guess is zero.
- *
- * \return      In most cases, a small positive integer is returned;
- *              for optimal use, the encoded output of this filter
- *              chain should start at on offset that is a multiple of
- *              the returned integer value.
- *
- *              If the alignment of the output is the same as the input
- *              data (which this function cannot know), \a guess is
- *              returned.
- *
- *              If an error occurs (that is, unknown Filter IDs or filter
- *              options), UINT32_MAX is returned.
- */
-extern uint32_t lzma_alignment_output(
-		const lzma_options_filter *filters, uint32_t guess);

src/liblzma/api/lzma/alone.h

@@ -1,82 +0,0 @@
-/**
- * \file        lzma/alone.h
- * \brief       Handling of the legacy LZMA_Alone format
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Options for files in the LZMA_Alone format
- */
-typedef struct {
-	/**
-	 * \brief       Uncompressed Size and usage of End of Payload Marker
-	 *
-	 * In contrast to .lzma Blocks, LZMA_Alone format cannot have both
-	 * uncompressed size field in the header and end of payload marker.
-	 * If you don't know the uncompressed size beforehand, set it to
-	 * LZMA_VLI_VALUE_UNKNOWN and liblzma will embed end of payload
-	 * marker.
-	 */
-	lzma_vli uncompressed_size;
-
-	/**
-	 * \brief       LZMA options
-	 *
-	 * The LZMA_Alone format supports only one filter: the LZMA filter.
-	 *
-	 * \note        There exists also an undocumented variant of the
-	 *              LZMA_Alone format, which uses the x86 filter in
-	 *              addition to LZMA. This format was never supported
-	 *              by LZMA Utils and is not supported by liblzma either.
-	 */
-	lzma_options_lzma lzma;
-
-} lzma_options_alone;
-
-
-/**
- * \brief       Initializes LZMA_Alone encoder
- *
- * LZMA_Alone files have the suffix .lzma like the .lzma Stream files.
- * LZMA_Alone format supports only one filter, the LZMA filter. There is
- * no support for integrity checks like CRC32.
- *
- * Use this format if and only if you need to create files readable by
- * legacy LZMA tools.
- *
- * LZMA_Alone encoder doesn't support LZMA_SYNC_FLUSH or LZMA_FULL_FLUSH.
- *
- * \return      - LZMA_OK
- *              - LZMA_MEM_ERROR
- *              - LZMA_PROG_ERROR
- */
-extern lzma_ret lzma_alone_encoder(
-		lzma_stream *strm, const lzma_options_alone *options);
-
-
-/**
- * \brief       Initializes decoder for LZMA_Alone file
- *
- * The LZMA_Alone decoder supports LZMA_SYNC_FLUSH.
- *
- * \return      - LZMA_OK
- *              - LZMA_MEM_ERROR
- */
-extern lzma_ret lzma_alone_decoder(lzma_stream *strm);

src/liblzma/api/lzma/auto.h

@@ -1,41 +0,0 @@
-/**
- * \file        lzma/auto.h
- * \brief       Decoder with automatic file format detection
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Decode .lzma Streams and LZMA_Alone files with autodetection
- *
- * Autodetects between the .lzma Stream and LZMA_Alone formats, and
- * calls lzma_stream_decoder_init() or lzma_alone_decoder_init() once
- * the type of the file has been detected.
- *
- * \param       strm        Pointer to propertily prepared lzma_stream
- * \param       header      Pointer to hold a pointer to Extra Records read
- *                          from the Header Metadata Block. Use NULL if
- *                          you don't care about Extra Records.
- * \param       footer      Same as header, but for Footer Metadata Block.
- *
- * \return      - LZMA_OK: Initialization was successful.
- *              - LZMA_MEM_ERROR: Cannot allocate memory.
- */
-extern lzma_ret lzma_auto_decoder(lzma_stream *strm,
-		lzma_extra **header, lzma_extra **footer);

src/liblzma/api/lzma/base.h

@@ -33,28 +33,193 @@
 typedef unsigned char lzma_bool;
 
 
+/**
+ * \brief       Type of reserved enumeration variable in structures
+ *
+ * To avoid breaking library ABI when new features are added, several
+ * structures contain extra variables that may be used in future. Since
+ * sizeof(enum) can be different than sizeof(int), and sizeof(enum) may
+ * even vary depending on the range of enumeration constants, we specify
+ * a separate type to be used for reserved enumeration variables. All
+ * enumeration constants in liblzma API will be non-negative and less
+ * than 128, which should guarantee that the ABI won't break even when
+ * new constants are added to existing enumerations.
+ */
+typedef enum {
+	LZMA_RESERVED_ENUM      = 0
+} lzma_reserved_enum;
+
+
 /**
  * \brief       Return values used by several functions in liblzma
  *
  * Check the descriptions of specific functions to find out which return
- * values they can return and the exact meanings of the values in every
- * situation. The descriptions given here are only suggestive.
+ * values they can return. With some functions the return values may have
+ * more specific meanings than described here; those differences are
+ * described per-function basis.
  */
 typedef enum {
-	LZMA_OK                 =  0,
+	LZMA_OK                 = 0,
 		/**<
 		 * \brief       Operation completed successfully
 		 */
 
-	LZMA_STREAM_END         =  1,
+	LZMA_STREAM_END         = 1,
 		/**<
 		 * \brief       End of stream was reached
 		 *
-		 * The application should pick the last remaining output
-		 * bytes from strm->next_out.
+		 * In encoder, LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, or
+		 * LZMA_FINISH was finished. In decoder, this indicates
+		 * that all the data was successfully decoded.
+		 *
+		 * In all cases, when LZMA_STREAM_END is returned, the last
+		 * output bytes should be picked from strm->next_out.
 		 */
 
-	LZMA_PROG_ERROR       = -2,
+	LZMA_NO_CHECK           = 2,
+		/**<
+		 * \brief       Input stream has no integrity check
+		 *
+		 * This return value can be returned only if the
+		 * LZMA_TELL_NO_CHECK flag was used when initializing
+		 * the decoder. LZMA_NO_CHECK is just a warning, and
+		 * the decoding can be continued normally.
+		 *
+		 * It is possible to call lzma_get_check() immediatelly after
+		 * lzma_code has returned LZMA_NO_CHECK. The result will
+		 * naturally be LZMA_CHECK_NONE, but the possibility to call
+		 * lzma_get_check() may be convenient in some applications.
+		 */
+
+	LZMA_UNSUPPORTED_CHECK  = 3,
+		/**<
+		 * \brief       Cannot calculate the integrity check
+		 *
+		 * The usage of this return value is different in encoders
+		 * and decoders.
+		 *
+		 * Encoders can return this value only from the initialization
+		 * function. If initialization fails with this value, the
+		 * encoding cannot be done, because there's no way to produce
+		 * output with the correct integrity check.
+		 *
+		 * Decoders can return this value only from lzma_code() and
+		 * only if the LZMA_TELL_UNSUPPORTED_CHECK flag was used when
+		 * initializing the decoder. The decoding can still be
+		 * continued normally even if the check type is unsupported,
+		 * but naturally the check will not be validated, and possible
+		 * errors may go undetected.
+		 *
+		 * With decoder, it is possible to call lzma_get_check()
+		 * immediatelly after lzma_code() has returned
+		 * LZMA_UNSUPPORTED_CHECK. This way it is possible to find
+		 * out what the unsupported Check ID was.
+		 */
+
+	LZMA_GET_CHECK          = 4,
+		/**<
+		 * \brief       Integrity check type is now available
+		 *
+		 * This value can be returned only by the lzma_code() function
+		 * and only if the decoder was initialized with the
+		 * LZMA_TELL_ANY_CHECK flag. LZMA_GET_CHECK tells the
+		 * application that it may now call lzma_get_check() to find
+		 * out the Check ID. This can be used, for example, to
+		 * implement a decoder that accepts only files that have
+		 * strong enough integrity check.
+		 */
+
+	LZMA_MEM_ERROR          = 5,
+		/**<
+		 * \brief       Cannot allocate memory
+		 *
+		 * Memory allocation failed, or the size of the allocation
+		 * would be greater than SIZE_MAX.
+		 *
+		 * Due to internal implementation reasons, the coding cannot
+		 * be continued even if more memory were made available after
+		 * LZMA_MEM_ERROR.
+		 */
+
+	LZMA_MEMLIMIT_ERROR     = 6,
+		/**
+		 * \brief       Memory usage limit was reached
+		 *
+		 * Decoder would need more memory than allowed by the
+		 * specified memory usage limit. To continue decoding,
+		 * the memory usage limit has to be increased with
+		 * lzma_memlimit().
+		 */
+
+	LZMA_FORMAT_ERROR       = 7,
+		/**<
+		 * \brief       File format not recognized
+		 *
+		 * The decoder did not recognize the input as supported file
+		 * format. This error can occur, for example, when trying to
+		 * decode .lzma format file with lzma_stream_decoder,
+		 * because lzma_stream_decoder accepts only the .xz format.
+		 */
+
+	LZMA_OPTIONS_ERROR      = 8,
+		/**<
+		 * \brief       Invalid or unsupported options
+		 *
+		 * Invalid or unsupported options, for example
+		 *  - unsupported filter(s) or filter options; or
+		 *  - reserved bits set in headers (decoder only).
+		 *
+		 * Rebuilding liblzma with more features enabled, or
+		 * upgrading to a newer version of liblzma may help.
+		 */
+
+	LZMA_DATA_ERROR         = 9,
+		/**<
+		 * \brief       Data is corrupt
+		 *
+		 * The usage of this return value is different in encoders
+		 * and decoders. In both encoder and decoder, the coding
+		 * cannot continue after this error.
+		 *
+		 * Encoders return this if size limits of the target file
+		 * format would be exceeded. These limits are huge, thus
+		 * getting this error from an encoder is mostly theoretical.
+		 * For example, the maximum compressed and uncompressed
+		 * size of a .xz Stream created with lzma_stream_encoder is
+		 * 2^63 - 1 bytes (one byte less than 8 EiB).
+		 *
+		 * Decoders return this error if the input data is corrupt.
+		 * This can mean, for example, invalid CRC32 in headers
+		 * or invalid check of uncompressed data.
+		 */
+
+	LZMA_BUF_ERROR          = 10,
+		/**<
+		 * \brief       No progress is possible
+		 *
+		 * This error code is returned when the coder cannot consume
+		 * any new input and produce any new output. The most common
+		 * reason for this error is that the input stream being
+		 * decoded is truncated or corrupt.
+		 *
+		 * This error is not fatal. Coding can be continued normally
+		 * by providing more input and/or more output space, if
+		 * possible.
+		 *
+		 * Typically the first call to lzma_code() that can do no
+		 * progress returns LZMA_OK instead of LZMA_BUF_ERROR. Only
+		 * the second consecutive call doing no progress will return
+		 * LZMA_BUF_ERROR. This is intentional.
+		 *
+		 * With zlib, Z_BUF_ERROR may be returned even if the
+		 * application is doing nothing wrong. The above hack
+		 * guarantees that liblzma never returns LZMA_BUF_ERROR
+		 * to properly written applications unless the input file
+		 * is truncated or corrupt. This should simplify the
+		 * applications a little.
+		 */
+
+	LZMA_PROG_ERROR         = 11,
 		/**<
 		 * \brief       Programming error
 		 *
@@ -73,70 +238,24 @@ typedef enum {
 		 * can be a sign of a bug in liblzma. See the documentation
 		 * how to report bugs.
 		 */
-
-	LZMA_DATA_ERROR         = -3,
-		/**<
-		 * \brief       Data is corrupt
-		 *
-		 * - Encoder: The input size doesn't match the uncompressed
-		 *   size given to lzma_*_encoder_init().
-		 * - Decoder: The input is corrupt. This includes corrupted
-		 *   header, corrupted compressed data, and unmatching
-		 *   integrity Check.
-		 *
-		 * \todo        What can be done if encoder returns this?
-		 *              Probably can continue by fixing the input
-		 *              amount, but make sure.
-		 */
-
-	LZMA_MEM_ERROR          = -4,
-		/**<
-		 * \brief       Cannot allocate memory
-		 *
-		 * Memory allocation failed.
-		 */
-
-	LZMA_BUF_ERROR          = -5,
-		/**<
-		 * \brief       No progress is possible
-		 *
-		 * This may happen when avail_in or avail_out is zero.
-		 *
-		 * \note        This error is not fatal. Coding can continue
-		 *              normally once the reason for this error has
-		 *              been fixed.
-		 */
-
-	LZMA_HEADER_ERROR       = -6,
-		/**<
-		 * \brief       Invalid or unsupported header
-		 *
-		 * Invalid or unsupported options, for example
-		 *  - unsupported filter(s) or filter options; or
-		 *  - reserved bits set in headers (decoder only).
-		 *
-		 * Rebuilding liblzma with more features enabled, or
-		 * upgrading to a newer version of liblzma may help.
-		 */
-
-	LZMA_UNSUPPORTED_CHECK  = -7,
-		/**<
-		 * \brief       Check type is unknown
-		 *
-		 * The type of Check is not supported, and thus the Check
-		 * cannot be calculated. In the encoder, this is an error.
-		 * In the decoder, this is only a warning and decoding can
-		 * still proceed normally (but the Check is ignored).
-		 */
 } lzma_ret;
 
 
 /**
  * \brief       The `action' argument for lzma_code()
+ *
+ * After the first use of LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, or LZMA_FINISH,
+ * the same `action' must is used until lzma_code() returns LZMA_STREAM_END.
+ * Also, the amount of input (that is, strm->avail_in) must not be modified
+ * by the application until lzma_code() returns LZMA_STREAM_END. Changing the
+ * `action' or modifying the amount of input will make lzma_code() return
+ * LZMA_PROG_ERROR.
  */
 typedef enum {
 	LZMA_RUN = 0,
 		/**<
+		 * \brief       Continue coding
+		 *
 		 * Encoder: Encode as much input as possible. Some internal
 		 * buffering will probably be done (depends on the filter
 		 * chain in use), which causes latency: the input used won't
@@ -144,55 +263,65 @@ typedef enum {
 		 * lzma_code() call.
 		 *
 		 * Decoder: Decode as much input as possible and produce as
-		 * much output as possible. This action provides best
-		 * throughput, but may introduce latency, because the
-		 * decoder may decode more data into its internal buffers
-		 * than that fits into next_out.
+		 * much output as possible.
 		 */
 
 	LZMA_SYNC_FLUSH = 1,
 		/**<
-		 * Encoder: Makes all the data given to liblzma via next_in
-		 * available in next_out without resetting the filters. Call
-		 * lzma_code() with LZMA_SYNC_FLUSH until it returns
-		 * LZMA_STREAM_END. Then continue encoding normally.
+		 * \brief       Make all the input available at output
 		 *
-		 * \note        Synchronous flushing is supported only by
-		 *              some filters. Some filters support it only
-		 *              partially.
+		 * Normally the encoder introduces some latency.
+		 * LZMA_SYNC_FLUSH forces all the buffered data to be
+		 * available at output without resetting the internal
+		 * state of the encoder. This way it is possible to use
+		 * compressed stream for example for communication over
+		 * network.
 		 *
-		 * Decoder: Asks the decoder to decode only as much as is
-		 * needed to fill next_out. This decreases latency with some
-		 * filters, but is likely to decrease also throughput. It is
-		 * a good idea to use this flag only when it is likely that
-		 * you don't need more output soon.
+		 * Only some filters support LZMA_SYNC_FLUSH. Trying to use
+		 * LZMA_SYNC_FLUSH with filters that don't support it will
+		 * make lzma_code() return LZMA_OPTIONS_ERROR. For example,
+		 * LZMA1 doesn't support LZMA_SYNC_FLUSH but LZMA2 does.
 		 *
-		 * \note        With decoder, this is not comparable to
-		 *              zlib's Z_SYNC_FLUSH.
+		 * Using LZMA_SYNC_FLUSH very often can dramatically reduce
+		 * the compression ratio. With some filters (for example,
+		 * LZMA2), finetuning the compression options may help
+		 * mitigate this problem significantly.
+		 *
+		 * Decoders don't support LZMA_SYNC_FLUSH.
 		 */
 
 	LZMA_FULL_FLUSH = 2,
 		/**<
-		 * Finishes encoding of the current Data Block. All the input
-		 * data going to the current Data Block must have been given
+		 * \brief       Make all the input available at output
+		 *
+		 * Finish encoding of the current Block. All the input
+		 * data going to the current Block must have been given
 		 * to the encoder (the last bytes can still be pending in
 		 * next_in). Call lzma_code() with LZMA_FULL_FLUSH until
 		 * it returns LZMA_STREAM_END. Then continue normally with
 		 * LZMA_RUN or finish the Stream with LZMA_FINISH.
 		 *
-		 * This action is supported only by Multi-Block Stream
-		 * encoder. If there is no unfinished Data Block, no empty
-		 * Data Block is created.
+		 * This action is currently supported only by Stream encoder
+		 * and easy encoder (which uses Stream encoder). If there is
+		 * no unfinished Block, no empty Block is created.
 		 */
 
 	LZMA_FINISH = 3
 		/**<
-		 * Finishes the encoding operation. All the input data must
+		 * \brief       Finish the coding operation
+		 *
+		 * Finishes the coding operation. All the input data must
 		 * have been given to the encoder (the last bytes can still
 		 * be pending in next_in). Call lzma_code() with LZMA_FINISH
-		 * until it returns LZMA_STREAM_END.
+		 * until it returns LZMA_STREAM_END. Once LZMA_FINISH has
+		 * been used, the amount of input must no longer be changed
+		 * by the application.
 		 *
-		 * This action is not supported by decoders.
+		 * When decoding, using LZMA_FINISH is optional unless the
+		 * LZMA_CONCATENATED flag was used when the decoder was
+		 * initialized. When LZMA_CONCATENATED was not used, the only
+		 * effect of LZMA_FINISH is that the amount of input must not
+		 * be changed just like in the encoder.
 		 */
 } lzma_action;
 
@@ -201,8 +330,10 @@ typedef enum {
  * \brief       Custom functions for memory handling
  *
  * A pointer to lzma_allocator may be passed via lzma_stream structure
- * to liblzma. The library will use these functions for memory handling
- * instead of the default malloc() and free().
+ * to liblzma, and some advanced functions take a pointer to lzma_allocator
+ * as a separate function argument. The library will use the functions
+ * specified in lzma_allocator for memory handling instead of the default
+ * malloc() and free().
  *
  * liblzma doesn't make an internal copy of lzma_allocator. Thus, it is
  * OK to change these function pointers in the middle of the coding
@@ -212,47 +343,51 @@ typedef enum {
  */
 typedef struct {
 	/**
-	 * \brief       Pointer to custom memory allocation function
-	 *
-	 * Set this to point to your custom memory allocation function.
-	 * It can be useful for example if you want to limit how much
-	 * memory liblzma is allowed to use: for this, you may use
-	 * a pointer to lzma_memory_alloc().
+	 * \brief       Pointer to a custom memory allocation function
 	 *
 	 * If you don't want a custom allocator, but still want
 	 * custom free(), set this to NULL and liblzma will use
 	 * the standard malloc().
 	 *
 	 * \param       opaque  lzma_allocator.opaque (see below)
-	 * \param       nmemb   Number of elements like in calloc().
-	 *                      liblzma will always set nmemb to 1.
-	 *                      This argument exists only for
+	 * \param       nmemb   Number of elements like in calloc(). liblzma
+	 *                      will always set nmemb to 1, so it is safe to
+	 *                      ignore nmemb in a custom allocator if you like.
+	 *                      The nmemb argument exists only for
 	 *                      compatibility with zlib and libbzip2.
 	 * \param       size    Size of an element in bytes.
 	 *                      liblzma never sets this to zero.
 	 *
 	 * \return      Pointer to the beginning of a memory block of
-	 *              size nmemb * size, or NULL if allocation fails
+	 *              `size' bytes, or NULL if allocation fails
 	 *              for some reason. When allocation fails, functions
 	 *              of liblzma return LZMA_MEM_ERROR.
+	 *
+	 * For performance reasons, the allocator should not waste time
+	 * zeroing the allocated buffers. This is not only about speed, but
+	 * also memory usage, since the operating system kernel doesn't
+	 * necessarily allocate the requested memory in physical memory until
+	 * it is actually used. With small input files liblzma may actually
+	 * need only a fraction of the memory that it requested for allocation.
+	 *
+	 * \note        LZMA_MEM_ERROR is also used when the size of the
+	 *              allocation would be greater than SIZE_MAX. Thus,
+	 *              don't assume that the custom allocator must have
+	 *              returned NULL if some function from liblzma
+	 *              returns LZMA_MEM_ERROR.
 	 */
 	void *(*alloc)(void *opaque, size_t nmemb, size_t size);
 
 	/**
-	 * \brief       Pointer to custom memory freeing function
-	 *
-	 * Set this to point to your custom memory freeing function.
-	 * If lzma_memory_alloc() is used as allocator, this should
-	 * be set to lzma_memory_free().
+	 * \brief       Pointer to a custom memory freeing function
 	 *
 	 * If you don't want a custom freeing function, but still
 	 * want a custom allocator, set this to NULL and liblzma
 	 * will use the standard free().
 	 *
 	 * \param       opaque  lzma_allocator.opaque (see below)
-	 * \param       ptr     Pointer returned by
-	 *                      lzma_allocator.alloc(), or when it
-	 *                      is set to NULL, a pointer returned
+	 * \param       ptr     Pointer returned by lzma_allocator.alloc(),
+	 *                      or when it is set to NULL, a pointer returned
 	 *                      by the standard malloc().
 	 */
 	void (*free)(void *opaque, void *ptr);
@@ -264,11 +399,7 @@ typedef struct {
 	 * and lzma_allocator.free(). This intended to ease implementing
 	 * custom memory allocation functions for use with liblzma.
 	 *
-	 * When using lzma_memory_alloc() and lzma_memory_free(), opaque
-	 * must point to lzma_memory_limitter structure allocated and
-	 * initialized with lzma_memory_limitter_create().
-	 *
-	 * If you don't need this, you should set it to NULL.
+	 * If you don't need this, you should set this to NULL.
 	 */
 	void *opaque;
 
@@ -291,31 +422,39 @@ typedef struct lzma_internal_s lzma_internal;
  *   - defining custom memory hander functions; and
  *   - holding a pointer to coder-specific internal data structures.
  *
- * Before calling any of the lzma_*_init() functions the first time,
- * the application must reset lzma_stream to LZMA_STREAM_INIT. The
- * lzma_*_init() function will verify the options, allocate internal
- * data structures and store pointer to them into `internal'. Finally
- * total_in and total_out are reset to zero. In contrast to zlib,
- * next_in and avail_in are ignored by the initialization functions.
+ * Typical usage:
  *
- * The actual coding is done with the lzma_code() function. Application
- * must update next_in, avail_in, next_out, and avail_out between
- * calls to lzma_decode() just like with zlib.
+ *  - After allocating lzma_stream (on stack or with malloc()), it must be
+ *    initialized to LZMA_STREAM_INIT (see LZMA_STREAM_INIT for details).
  *
- * In contrast to zlib, even the decoder requires that there always
- * is at least one byte space in next_out; if avail_out == 0,
- * LZMA_BUF_ERROR is returned immediatelly. This shouldn't be a problem
- * for most applications that already use zlib, but it's still worth
- * checking your application.
+ *  - Initialize a coder to the lzma_stream, for example by using
+ *    lzma_easy_encoder() or lzma_auto_decoder(). Some notes:
+ *      - In contrast to zlib, strm->next_in and strm->next_out are
+ *        ignored by all initialization functions, thus it is safe
+ *        to not initialize them yet.
+ *      - The initialization functions always set strm->total_in and
+ *        strm->total_out to zero.
+ *      - If the initialization function fails, no memory is left allocated
+ *        that would require freeing with lzma_end() even if some memory was
+ *        associated with the lzma_stream structure when the initialization
+ *        function was called.
  *
- * Application may modify values of total_in and total_out as it wants.
+ *  - Use lzma_code() to do the actual work.
+ *
+ *  - Once the coding has been finished, the existing lzma_stream can be
+ *    reused. It is OK to reuse lzma_stream with different initialization
+ *    function without calling lzma_end() first. Old allocations are
+ *    automatically freed.
+ *
+ *  - Finally, use lzma_end() to free the allocated memory. lzma_end() never
+ *    frees the lzma_stream structure itself.
+ *
+ * Application may modify the values of total_in and total_out as it wants.
  * They are updated by liblzma to match the amount of data read and
- * written, but liblzma doesn't use the values internally.
- *
- * Application must not touch the `internal' pointer.
+ * written, but aren't used for anything else.
  */
 typedef struct {
-	uint8_t *next_in;   /**< Pointer to the next input byte. */
+	const uint8_t *next_in; /**< Pointer to the next input byte. */
 	size_t avail_in;    /**< Number of available input bytes in next_in. */
 	uint64_t total_in;  /**< Total number of bytes read by liblzma. */
 
@@ -329,9 +468,22 @@ typedef struct {
 	 */
 	lzma_allocator *allocator;
 
-	/** Internal state is not visible to outsiders. */
+	/** Internal state is not visible to applications. */
 	lzma_internal *internal;
 
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. Excluding the initialization of this structure,
+	 * you should not touch these, because the names of these variables
+	 * may change.
+	 */
+	void *reserved_ptr1;
+	void *reserved_ptr2;
+	uint64_t reserved_int1;
+	uint64_t reserved_int2;
+	lzma_reserved_enum reserved_enum1;
+	lzma_reserved_enum reserved_enum2;
+
 } lzma_stream;
 
 
@@ -343,68 +495,101 @@ typedef struct {
  * has been allocated yet:
  *
  *     lzma_stream strm = LZMA_STREAM_INIT;
+ *
+ * If you need to initialize a dynamically allocated lzma_stream, you can use
+ * memset(strm_pointer, 0, sizeof(lzma_stream)). Strictly speaking, this
+ * violates the C standard since NULL may have different internal
+ * representation than zero, but it should be portable enough in practice.
+ * Anyway, for maximum portability, you can use something like this:
+ *
+ *     lzma_stream tmp = LZMA_STREAM_INIT;
+ *     *strm = tmp;
  */
-#define LZMA_STREAM_INIT { NULL, 0, 0, NULL, 0, 0, NULL, NULL }
+#define LZMA_STREAM_INIT \
+	{ NULL, 0, 0, NULL, 0, 0, NULL, NULL, \
+	NULL, NULL, 0, 0, LZMA_RESERVED_ENUM, LZMA_RESERVED_ENUM }
 
 
 /**
- * \brief       Initialization for lzma_stream
- *
- * This is like LZMA_STREAM_INIT, but this can be used when the lzma_stream
- * has already been allocated:
- *
- *     lzma_stream *strm = malloc(sizeof(lzma_stream));
- *     if (strm == NULL)
- *         return LZMA_MEM_ERROR;
- *     *strm = LZMA_STREAM_INIT_VAR;
- */
-extern const lzma_stream LZMA_STREAM_INIT_VAR;
-
-
-/**
- * \brief       Encodes or decodes data
+ * \brief       Encode or decode data
  *
  * Once the lzma_stream has been successfully initialized (e.g. with
- * lzma_stream_encoder_single()), the actual encoding or decoding is
- * done using this function.
+ * lzma_stream_encoder()), the actual encoding or decoding is done
+ * using this function. The application has to update strm->next_in,
+ * strm->avail_in, strm->next_out, and strm->avail_out to pass input
+ * to and get output from liblzma.
  *
- * \return      Some coders may have more exact meaning for different return
- *              values, which are mentioned separately in the description of
- *              the initialization functions. Here are the typical meanings:
- *              - LZMA_OK: So far all good.
- *              - LZMA_STREAM_END:
- *                  - Encoder: LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, or
- *                    LZMA_FINISH completed.
- *                  - Decoder: End of uncompressed data was reached.
- *              - LZMA_BUF_ERROR: Unable to progress. Provide more input or
- *                output space, and call this function again. This cannot
- *                occur if both avail_in and avail_out were non-zero (or
- *                there's a bug in liblzma).
- *              - LZMA_MEM_ERROR: Unable to allocate memory. Due to lazy
- *                programming, the coding cannot continue even if the
- *                application could free more memory. The next call must
- *                be lzma_end() or some initialization function.
- *              - LZMA_DATA_ERROR:
- *                  - Encoder: Filter(s) cannot process the given data.
- *                  - Decoder: Compressed data is corrupt.
- *              - LZMA_HEADER_ERROR: Unsupported options. Rebuilding liblzma
- *                with more features enabled or upgrading to a newer version
- *                may help, although usually this is a sign of invalid options
- *                (encoder) or corrupted input data (decoder).
- *              - LZMA_PROG_ERROR: Invalid arguments or the internal state
- *                of the coder is corrupt.
+ * See the description of the coder-specific initialization function to find
+ * out what `action' values are supported by the coder. See documentation of
+ * lzma_ret for the possible return values.
  */
-extern lzma_ret lzma_code(lzma_stream *strm, lzma_action action);
+extern lzma_ret lzma_code(lzma_stream *strm, lzma_action action)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Frees memory allocated for the coder data structures
+ * \brief       Free memory allocated for the coder data structures
  *
  * \param       strm    Pointer to lzma_stream that is at least initialized
  *                      with LZMA_STREAM_INIT.
  *
+ * After lzma_end(strm), strm->internal is guaranteed to be NULL. No other
+ * members of the lzma_stream structure are touched.
+ *
  * \note        zlib indicates an error if application end()s unfinished
- *              stream. liblzma doesn't do this, and assumes that
+ *              stream structure. liblzma doesn't do this, and assumes that
  *              application knows what it is doing.
  */
 extern void lzma_end(lzma_stream *strm);
+
+
+/**
+ * \brief       Get the memory usage of decoder filter chain
+ *
+ * This function is currently supported only when *strm has been initialized
+ * with a function that takes a memlimit argument. With other functions, you
+ * should use e.g. lzma_raw_encoder_memusage() or lzma_raw_decoder_memusage()
+ * to estimate the memory requirements.
+ *
+ * This function is useful e.g. after LZMA_MEMLIMIT_ERROR to find out how big
+ * the memory usage limit should have been to decode the input. Note that
+ * this may give misleading information if decoding .xz Streams that have
+ * multiple Blocks, because each Block can have different memory requirements.
+ *
+ * \return      Rough estimate of how much memory is currently allocated
+ *              for the filter decoders. If no filter chain is currently
+ *              allocated, some non-zero value is still returned, which is
+ *              less than or equal to what any filter chain would indicate
+ *              as its memory requirement.
+ *
+ *              If this function isn't supported by *strm or some other error
+ *              occurs, zero is returned.
+ */
+extern uint64_t lzma_memusage(const lzma_stream *strm);
+
+
+/**
+ * \brief       Get the current memory usage limit
+ *
+ * This function is supported only when *strm has been initialized with
+ * a function that takes a memlimit argument.
+ *
+ * \return      On success, the current memory usage limit is returned
+ *              (always non-zero). On error, zero is returned.
+ */
+extern uint64_t lzma_memlimit_get(const lzma_stream *strm);
+
+
+/**
+ * \brief       Set the memory usage limit
+ *
+ * This function is supported only when *strm has been initialized with
+ * a function that takes a memlimit argument.
+ *
+ * \return      - LZMA_OK: New memory usage limit successfully set.
+ *              - LZMA_MEMLIMIT_ERROR: The new limit is too small.
+ *                The limit was not changed.
+ *              - LZMA_PROG_ERROR: Invalid arguments, e.g. *strm doesn't
+ *                support memory usage limit or memlimit was zero.
+ */
+extern lzma_ret lzma_memlimit_set(lzma_stream *strm, uint64_t memlimit);

src/liblzma/api/lzma/bcj.h

@@ -1,6 +1,6 @@
 /**
- * \file        lzma/simple.h
- * \brief       So called "simple" filters
+ * \file        lzma/bcj.h
+ * \brief       Branch/Call/Jump conversion filters
  *
  * \author      Copyright (C) 1999-2006 Igor Pavlov
  * \author      Copyright (C) 2007 Lasse Collin
@@ -21,11 +21,11 @@
 #endif
 
 
-/* Filter IDs for lzma_options_filter.id */
+/* Filter IDs for lzma_filter.id */
 
 #define LZMA_FILTER_X86         LZMA_VLI_C(0x04)
 	/**<
-	 * BCJ (Branch, Call, Jump) filter for x86 binaries
+	 * Filter for x86 binaries
 	 */
 
 #define LZMA_FILTER_POWERPC     LZMA_VLI_C(0x05)
@@ -55,19 +55,28 @@
 
 
 /**
- * \brief       Options for so called "simple" filters
+ * \brief       Options for BCJ filters
  *
- * The simple filters never change the size of the data. Specifying options
- * for them is optional: if pointer to options is NULL, default values are
- * used. You probably never need to specify these options, so just set the
- * options pointer to NULL and be happy.
+ * The BCJ filters never change the size of the data. Specifying options
+ * for them is optional: if pointer to options is NULL, default value is
+ * used. You probably never need to specify options to BCJ filters, so just
+ * set the options pointer to NULL and be happy.
  *
  * If options with non-default values have been specified when encoding,
  * the same options must also be specified when decoding.
+ *
+ * \note        At the moment, none of the BCJ filters support
+ *              LZMA_SYNC_FLUSH. If LZMA_SYNC_FLUSH is specified,
+ *              LZMA_OPTIONS_ERROR will be returned. If there is need,
+ *              partial support for LZMA_SYNC_FLUSH can be added in future.
+ *              Partial means that flushing would be possible only at
+ *              offsets that are multiple of 2, 4, or 16 depending on
+ *              the filter, except x86 which cannot be made to support
+ *              LZMA_SYNC_FLUSH predictably.
  */
 typedef struct {
 	/**
-	 * \brief       Start offset for branch conversions
+	 * \brief       Start offset for conversions
 	 *
 	 * This setting is useful only when the same filter is used
 	 * _separately_ for multiple sections of the same executable file,
@@ -77,9 +86,9 @@ typedef struct {
 	 * of the cross-section branch/call/jump instructions will use the
 	 * same absolute addresses as in the first section.
 	 *
-	 * When the pointer to options is NULL, the default value is used.
-	 * The default value is zero.
+	 * When the pointer to options is NULL, the default value (zero)
+	 * is used.
 	 */
 	uint32_t start_offset;
 
-} lzma_options_simple;
+} lzma_options_bcj;

src/liblzma/api/lzma/block.h

@@ -1,6 +1,6 @@
 /**
  * \file        lzma/block.h
- * \brief       .lzma Block handling
+ * \brief       .xz Block handling
  *
  * \author      Copyright (C) 1999-2006 Igor Pavlov
  * \author      Copyright (C) 2007 Lasse Collin
@@ -22,10 +22,10 @@
 
 
 /**
- * \brief       Options for the Block Header encoder and decoder
+ * \brief       Options for the Block and Block Header encoders and decoders
  *
- * Different things use different parts of this structure. Some read
- * some members, other functions write, and some do both. Only the
+ * Different Block handling functions use different parts of this structure.
+ * Some read some members, other functions write, and some do both. Only the
  * members listed for reading need to be initialized when the specified
  * functions are called. The members marked for writing will be assigned
  * new values at some point either by calling the given function or by
@@ -33,113 +33,118 @@
  */
 typedef struct {
 	/**
-	 * \brief       Type of integrity Check
+	 * \brief       Block format version
 	 *
-	 * The type of the integrity Check is not stored into the Block
-	 * Header, thus its value must be provided also when decoding.
+	 * To prevent API and ABI breakages if new features are needed in
+	 * Block, a version number is used to indicate which fields in this
+	 * structure are in use. For now, version must always be zero.
+	 * With non-zero version, most Block related functions will return
+	 * LZMA_OPTIONS_ERROR.
 	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_check_type check;
-
-	/**
-	 * \brief       Precense of CRC32 of the Block Header
-	 *
-	 * Set this to true if CRC32 of the Block Header should be
-	 * calculated and stored in the Block Header.
-	 *
-	 * There is no way to autodetect if CRC32 is present in the Block
-	 * Header, thus this information must be provided also when decoding.
+	 * The decoding functions will always set this to the lowest value
+	 * that supports all the features indicated by the Block Header field.
+	 * The application must check that the version number set by the
+	 * decoding functions is supported by the application. Otherwise it
+	 * is possible that the application will decode the Block incorrectly.
 	 *
 	 * Read by:
 	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 *  - lzma_block_header_decoder()
-	 */
-	lzma_bool has_crc32;
-
-	/**
-	 * \brief       Usage of End of Payload Marker
-	 *
-	 * If this is true, End of Payload Marker is used even if
-	 * Uncompressed Size is known.
-	 *
-	 * Read by:
-	 *  - lzma_block_header_encoder()
+	 *  - lzma_block_header_encode()
+	 *  - lzma_block_compressed_size()
+	 *  - lzma_block_unpadded_size()
+	 *  - lzma_block_total_size()
 	 *  - lzma_block_encoder()
 	 *  - lzma_block_decoder()
 	 *
 	 * Written by:
-	 *  - lzma_block_header_decoder()
+	 *  - lzma_block_header_decode()
 	 */
-	lzma_bool has_eopm;
+	uint32_t version;
 
 	/**
-	 * \brief       True if the Block is a Metadata Block
+	 * \brief       Size of the Block Header field
 	 *
-	 * If this is true, the Metadata bit will be set in the Block Header.
-	 * It is up to the application to store correctly formatted data
-	 * into Metadata Block.
+	 * This is always a multiple of four.
 	 *
 	 * Read by:
-	 *  - lzma_block_header_encoder()
+	 *  - lzma_block_header_encode()
+	 *  - lzma_block_header_decode()
+	 *  - lzma_block_compressed_size()
+	 *  - lzma_block_unpadded_size()
+	 *  - lzma_block_total_size()
+	 *  - lzma_block_decoder()
 	 *
 	 * Written by:
-	 *  - lzma_block_header_decoder()
+	 *  - lzma_block_header_size()
 	 */
-	lzma_bool is_metadata;
+	uint32_t header_size;
+#	define LZMA_BLOCK_HEADER_SIZE_MIN 8
+#	define LZMA_BLOCK_HEADER_SIZE_MAX 1024
 
 	/**
-	 * \brief       True if Uncompressed Size is in Block Footer
+	 * \brief       Type of integrity Check
+	 *
+	 * The Check ID is not stored into the Block Header, thus its value
+	 * must be provided also when decoding.
 	 *
 	 * Read by:
+	 *  - lzma_block_header_encode()
+	 *  - lzma_block_header_decode()
+	 *  - lzma_block_compressed_size()
+	 *  - lzma_block_unpadded_size()
+	 *  - lzma_block_total_size()
 	 *  - lzma_block_encoder()
 	 *  - lzma_block_decoder()
 	 */
-	lzma_bool has_uncompressed_size_in_footer;
-
-	/**
-	 * \brief       True if Backward Size is in Block Footer
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_bool has_backward_size;
-
-	/**
-	 * \brief       True if Block coder should take care of Padding
-	 *
-	 * In liblzma, Stream decoder sets this to true when decoding
-	 * Header Metadata Block or Data Blocks from Multi-Block Stream,
-	 * and to false when decoding Single-Block Stream or Footer
-	 * Metadata Block from a Multi-Block Stream.
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_bool handle_padding;
+	lzma_check check;
 
 	/**
 	 * \brief       Size of the Compressed Data in bytes
 	 *
-	 * Usually you don't know this value when encoding in streamed mode.
-	 * In non-streamed mode you can reserve space for this field when
-	 * encoding the Block Header the first time, and then re-encode the
-	 * Block Header and copy it over the original one after the encoding
-	 * of the Block has been finished.
+	 * Encoding: If this is not LZMA_VLI_UNKNOWN, Block Header encoder
+	 * will store this value to the Block Header. Block encoder doesn't
+	 * care about this value, but will set it once the encoding has been
+	 * finished.
+	 *
+	 * Decoding: If this is not LZMA_VLI_UNKNOWN, Block decoder will
+	 * verify that the size of the Compressed Data field matches
+	 * compressed_size.
+	 *
+	 * Usually you don't know this value when encoding in streamed mode,
+	 * and thus cannot write this field into the Block Header.
+	 *
+	 * In non-streamed mode you can reserve space for this field before
+	 * encoding the actual Block. After encoding the data, finish the
+	 * Block by encoding the Block Header. Steps in detail:
+	 *
+	 *  - Set compressed_size to some big enough value. If you don't know
+	 *    better, use LZMA_VLI_MAX, but remember that bigger values take
+	 *    more space in Block Header.
+	 *
+	 *  - Call lzma_block_header_size() to see how much space you need to
+	 *    reserve for the Block Header.
+	 *
+	 *  - Encode the Block using lzma_block_encoder() and lzma_code().
+	 *    It sets compressed_size to the correct value.
+	 *
+	 *  - Use lzma_block_header_encode() to encode the Block Header.
+	 *    Because space was reserved in the first step, you don't need
+	 *    to call lzma_block_header_size() anymore, because due to
+	 *    reserving, header_size has to be big enough. If it is "too big",
+	 *    lzma_block_header_encode() will add enough Header Padding to
+	 *    make Block Header to match the size specified by header_size.
 	 *
 	 * Read by:
 	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 *  - lzma_block_encoder()
+	 *  - lzma_block_header_encode()
+	 *  - lzma_block_compressed_size()
+	 *  - lzma_block_unpadded_size()
+	 *  - lzma_block_total_size()
 	 *  - lzma_block_decoder()
 	 *
 	 * Written by:
-	 *  - lzma_block_header_decoder()
+	 *  - lzma_block_header_decode()
+	 *  - lzma_block_compressed_size()
 	 *  - lzma_block_encoder()
 	 *  - lzma_block_decoder()
 	 */
@@ -148,262 +153,265 @@ typedef struct {
 	/**
 	 * \brief       Uncompressed Size in bytes
 	 *
-	 * Encoder: If this value is not LZMA_VLI_VALUE_UNKNOWN, it is stored
-	 * to the Uncompressed Size field in the Block Header. The real
-	 * uncompressed size of the data being compressed must match
-	 * the Uncompressed Size or LZMA_HEADER_ERROR is returned.
+	 * This is handled very similarly to compressed_size above.
 	 *
-	 * If Uncompressed Size is unknown, End of Payload Marker must
-	 * be used. If uncompressed_size == LZMA_VLI_VALUE_UNKNOWN and
-	 * has_eopm == 0, LZMA_HEADER_ERROR will be returned.
-	 *
-	 * Decoder: If this value is not LZMA_VLI_VALUE_UNKNOWN, it is
-	 * compared to the real Uncompressed Size. If they do not match,
-	 * LZMA_HEADER_ERROR is returned.
+	 * Unlike compressed_size, uncompressed_size is needed by fewer
+	 * functions. This is because uncompressed_size isn't needed to
+	 * validate that Block stays within proper limits.
 	 *
 	 * Read by:
 	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 *  - lzma_block_encoder()
+	 *  - lzma_block_header_encode()
 	 *  - lzma_block_decoder()
 	 *
 	 * Written by:
-	 *  - lzma_block_header_decoder()
+	 *  - lzma_block_header_decode()
 	 *  - lzma_block_encoder()
 	 *  - lzma_block_decoder()
 	 */
 	lzma_vli uncompressed_size;
 
-	/**
-	 * \brief       Number of bytes to reserve for Compressed Size
-	 *
-	 * This is useful if you want to be able to store the Compressed Size
-	 * to the Block Header, but you don't know it when starting to encode.
-	 * Setting this to non-zero value at maximum of LZMA_VLI_BYTES_MAX,
-	 * the Block Header encoder will force the Compressed Size field to
-	 * occupy specified number of bytes. You can later rewrite the Block
-	 * Header to contain correct information by using otherwise identical
-	 * lzma_options_block structure except the correct compressed_size.
-	 *
-	 * Read by:
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 *
-	 * Written by:
-	 *  - lzma_block_header_decoder()
-	 */
-	uint32_t compressed_reserve;
-
-	/**
-	 * \brief       Number of bytes to reserve for Uncompressed Size
-	 *
-	 * See the description of compressed_size above.
-	 *
-	 * Read by:
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 *
-	 * Written by:
-	 *  - lzma_block_header_decoder()
-	 */
-	uint32_t uncompressed_reserve;
-
-	/**
-	 * \brief       Total Size of the Block in bytes
-	 *
-	 * This is useful in the decoder, which can verify the Total Size
-	 * if it is known from Index.
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 *
-	 * Written by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_vli total_size;
-
-	/**
-	 * \brief       Upper limit of Total Size
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_vli total_limit;
-
-	/**
-	 * \brief       Upper limit of Uncompressed Size
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 */
-	lzma_vli uncompressed_limit;
-
 	/**
 	 * \brief       Array of filters
 	 *
-	 * There can be at maximum of seven filters. The end of the array
-	 * is marked with .id = LZMA_VLI_VALUE_UNKNOWN. Minimum number of
-	 * filters is zero; in that case, an implicit Copy filter is used.
+	 * There can be 1-4 filters. The end of the array is marked with
+	 * .id = LZMA_VLI_UNKNOWN.
 	 *
 	 * Read by:
 	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
+	 *  - lzma_block_header_encode()
 	 *  - lzma_block_encoder()
 	 *  - lzma_block_decoder()
 	 *
 	 * Written by:
-	 *  - lzma_block_header_decoder(): Note that this does NOT free()
-	 *    the old filter options structures. If decoding fails, the
-	 *    caller must take care of freeing the options structures
-	 *    that may have been allocated and decoded before the error
-	 *    occurred.
+	 *  - lzma_block_header_decode(): Note that this does NOT free()
+	 *    the old filter options structures. All unused filters[] will
+	 *    have .id == LZMA_VLI_UNKNOWN and .options == NULL. If
+	 *    decoding fails, all filters[] are guaranteed to be
+	 *    LZMA_VLI_UNKNOWN and NULL.
+	 *
+	 * \note        Because of the array is terminated with
+	 *              .id = LZMA_VLI_UNKNOWN, the actual array must
+	 *              have LZMA_FILTERS_MAX + 1 members or the Block
+	 *              Header decoder will overflow the buffer.
 	 */
-	lzma_options_filter filters[8];
+	lzma_filter *filters;
 
-	/**
-	 * \brief       Size of the Padding field
-	 *
-	 * The Padding field exist to allow aligning the Compressed Data field
-	 * optimally in the Block. See lzma_options_stream.alignment in
-	 * stream.h for more information.
-	 *
-	 * If you want the Block Header encoder to automatically calculate
-	 * optimal size for the Padding field by looking at the information
-	 * in filters[], set this to LZMA_BLOCK_HEADER_PADDING_AUTO. In that
-	 * case, you must also set the aligmnet variable to tell the the
-	 * encoder the aligmnet of the beginning of the Block Header.
-	 *
-	 * The decoder never sets this to LZMA_BLOCK_HEADER_PADDING_AUTO.
-	 *
-	 * Read by:
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder(): Note that this doesn't
-	 *    accept LZMA_BLOCK_HEADER_PADDING_AUTO.
-	 *
-	 * Written by (these never set padding to
-	 * LZMA_BLOCK_HEADER_PADDING_AUTO):
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_decoder()
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. You should not touch these, because the names
+	 * of these variables may change. These are and will never be used
+	 * with the currently supported options, so it is safe to leave these
+	 * uninitialized.
 	 */
-	int32_t padding;
-#	define LZMA_BLOCK_HEADER_PADDING_AUTO (-1)
-#	define LZMA_BLOCK_HEADER_PADDING_MIN 0
-#	define LZMA_BLOCK_HEADER_PADDING_MAX 31
+	void *reserved_ptr1;
+	void *reserved_ptr2;
+	void *reserved_ptr3;
+	uint32_t reserved_int1;
+	uint32_t reserved_int2;
+	lzma_vli reserved_int3;
+	lzma_vli reserved_int4;
+	lzma_vli reserved_int5;
+	lzma_vli reserved_int6;
+	lzma_vli reserved_int7;
+	lzma_vli reserved_int8;
+	lzma_reserved_enum reserved_enum1;
+	lzma_reserved_enum reserved_enum2;
+	lzma_reserved_enum reserved_enum3;
+	lzma_reserved_enum reserved_enum4;
+	lzma_bool reserved_bool1;
+	lzma_bool reserved_bool2;
+	lzma_bool reserved_bool3;
+	lzma_bool reserved_bool4;
+	lzma_bool reserved_bool5;
+	lzma_bool reserved_bool6;
+	lzma_bool reserved_bool7;
+	lzma_bool reserved_bool8;
 
-	/**
-	 * \brief       Alignment of the beginning of the Block Header
-	 *
-	 * This variable is read only if padding has been set to
-	 * LZMA_BLOCK_HEADER_PADDING_AUTO.
-	 *
-	 * Read by:
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_encoder()
-	 */
-	uint32_t alignment;
-
-	/**
-	 * \brief       Size of the Block Header
-	 *
-	 * Read by:
-	 *  - lzma_block_encoder()
-	 *  - lzma_block_decoder()
-	 *
-	 * Written by:
-	 *  - lzma_block_header_size()
-	 *  - lzma_block_header_decoder()
-	 */
-	uint32_t header_size;
-
-} lzma_options_block;
+} lzma_block;
 
 
 /**
- * \brief       Calculates the size of Header Padding and Block Header
+ * \brief       Decode the Block Header Size field
+ *
+ * To decode Block Header using lzma_block_header_decode(), the size of the
+ * Block Header has to be known and stored into lzma_block.header_size.
+ * The size can be calculated from the first byte of a Block using this macro.
+ * Note that if the first byte is 0x00, it indicates beginning of Index; use
+ * this macro only when the byte is not 0x00.
+ *
+ * There is no encoding macro, because Block Header encoder is enough for that.
+ */
+#define lzma_block_header_size_decode(b) (((uint32_t)(b) + 1) * 4)
+
+
+/**
+ * \brief       Calculate Block Header Size
+ *
+ * Calculate the minimum size needed for the Block Header field using the
+ * settings specified in the lzma_block structure. Note that it is OK to
+ * increase the calculated header_size value as long as it is a multiple of
+ * four and doesn't exceed LZMA_BLOCK_HEADER_SIZE_MAX. Increasing header_size
+ * just means that lzma_block_header_encode() will add Header Padding.
  *
  * \return      - LZMA_OK: Size calculated successfully and stored to
- *                options->header_size.
- *              - LZMA_HEADER_ERROR: Unsupported filters or filter options.
- *              - LZMA_PROG_ERROR: Invalid options
+ *                block->header_size.
+ *              - LZMA_OPTIONS_ERROR: Unsupported version, filters or
+ *                filter options.
+ *              - LZMA_PROG_ERROR: Invalid values like compressed_size == 0.
  *
  * \note        This doesn't check that all the options are valid i.e. this
  *              may return LZMA_OK even if lzma_block_header_encode() or
- *              lzma_block_encoder() would fail.
+ *              lzma_block_encoder() would fail. If you want to validate the
+ *              filter chain, consider using lzma_memlimit_encoder() which as
+ *              a side-effect validates the filter chain.
  */
-extern lzma_ret lzma_block_header_size(lzma_options_block *options);
+extern lzma_ret lzma_block_header_size(lzma_block *block)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Encodes Block Header
+ * \brief       Encode Block Header
  *
- * Encoding of the Block options is done with a single call instead of
- * first initializing and then doing the actual work with lzma_code().
+ * The caller must have calculated the size of the Block Header already with
+ * lzma_block_header_size(). If larger value than the one calculated by
+ * lzma_block_header_size() is used, the Block Header will be padded to the
+ * specified size.
  *
  * \param       out         Beginning of the output buffer. This must be
- *                          at least options->header_size bytes.
- * \param       options     Block options to be encoded.
+ *                          at least block->header_size bytes.
+ * \param       block       Block options to be encoded.
  *
- * \return      - LZMA_OK: Encoding was successful. options->header_size
+ * \return      - LZMA_OK: Encoding was successful. block->header_size
  *                bytes were written to output buffer.
- *              - LZMA_HEADER_ERROR: Invalid or unsupported options.
- *              - LZMA_PROG_ERROR
+ *              - LZMA_OPTIONS_ERROR: Invalid or unsupported options.
+ *              - LZMA_PROG_ERROR: Invalid arguments, for example
+ *                block->header_size is invalid or block->filters is NULL.
  */
-extern lzma_ret lzma_block_header_encode(
-		uint8_t *out, const lzma_options_block *options);
+extern lzma_ret lzma_block_header_encode(const lzma_block *block, uint8_t *out)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Initializes Block Header decoder
+ * \brief       Decode Block Header
  *
- * Because the results of this decoder are placed into *options,
- * strm->next_in, strm->avail_in, and strm->total_in are not used.
+ * The size of the Block Header must have already been decoded with
+ * lzma_block_header_size_decode() macro and stored to block->header_size.
+ * block->filters must have been allocated, but not necessarily initialized.
+ * Possible existing filter options are _not_ freed.
  *
- * The only valid `action' with lzma_code() is LZMA_RUN.
+ * \param       block       Destination for block options with header_size
+ *                          properly initialized.
+ * \param       allocator   lzma_allocator for custom allocator functions.
+ *                          Set to NULL to use malloc() (and also free()
+ *                          if an error occurs).
+ * \param       in          Beginning of the input buffer. This must be
+ *                          at least block->header_size bytes.
  *
- * \return      - LZMA_OK: Encoding was successful. options->header_size
- *                bytes were written to output buffer.
- *              - LZMA_HEADER_ERROR: Invalid or unsupported options.
- *              - LZMA_PROG_ERROR
+ * \return      - LZMA_OK: Decoding was successful. block->header_size
+ *                bytes were read from the input buffer.
+ *              - LZMA_OPTIONS_ERROR: The Block Header specifies some
+ *                unsupported options such as unsupported filters.
+ *              - LZMA_DATA_ERROR: Block Header is corrupt, for example,
+ *                the CRC32 doesn't match.
+ *              - LZMA_PROG_ERROR: Invalid arguments, for example
+ *                block->header_size is invalid or block->filters is NULL.
  */
-extern lzma_ret lzma_block_header_decoder(
-		lzma_stream *strm, lzma_options_block *options);
+extern lzma_ret lzma_block_header_decode(lzma_block *block,
+		lzma_allocator *allocator, const uint8_t *in)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Initializes .lzma Block encoder
+ * \brief       Validate and set Compressed Size according to Unpadded Size
  *
- * This function is required for multi-thread encoding. It may also be
- * useful when implementing custom file formats.
+ * Block Header stores Compressed Size, but Index has Unpadded Size. If the
+ * application has already parsed the Index and is now decoding Blocks,
+ * it can calculate Compressed Size from Unpadded Size. This function does
+ * exactly that with error checking:
+ *
+ *  - Compressed Size calculated from Unpadded Size must be positive integer,
+ *    that is, Unpadded Size must be big enough that after Block Header and
+ *    Check fields there's still at least one byte for Compressed Size.
+ *
+ *  - If Compressed Size was present in Block Header, the new value
+ *    calculated from Unpadded Size is compared against the value
+ *    from Block Header.
+ *
+ * \note        This function must be called _after_ decoding the Block Header
+ *              field so that it can properly validate Compressed Size if it
+ *              was present in Block Header.
+ *
+ * \return      - LZMA_OK: block->compressed_size was set successfully.
+ *              - LZMA_DATA_ERROR: unpadded_size is too small compared to
+ *                block->header_size and lzma_check_size(block->check).
+ *              - LZMA_PROG_ERROR: Some values are invalid. For example,
+ *                block->header_size must be a multiple of four and
+ *                between 8 and 1024 inclusive.
+ */
+extern lzma_ret lzma_block_compressed_size(
+		lzma_block *block, lzma_vli unpadded_size)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Calculate Unpadded Size
+ *
+ * The Index field stores Unpadded Size and Uncompressed Size. The latter
+ * can be taken directly from the lzma_block structure after coding a Block,
+ * but Unpadded Size needs to be calculated from Block Header Size,
+ * Compressed Size, and size of the Check field. This is where this function
+ * is needed.
+ *
+ * \return      Unpadded Size on success, or zero on error.
+ */
+extern lzma_vli lzma_block_unpadded_size(const lzma_block *block)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Calculate the total encoded size of a Block
+ *
+ * This is equivalent to lzma_block_unpadded_size() except that the returned
+ * value includes the size of the Block Padding field.
+ *
+ * \return      On success, total encoded size of the Block. On error,
+ *              zero is returned.
+ */
+extern lzma_vli lzma_block_total_size(const lzma_block *block)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Initialize .xz Block encoder
+ *
+ * Valid actions for lzma_code() are LZMA_RUN, LZMA_SYNC_FLUSH (only if the
+ * filter chain supports it), and LZMA_FINISH.
  *
  * \return      - LZMA_OK: All good, continue with lzma_code().
  *              - LZMA_MEM_ERROR
- *              - LZMA_HEADER_ERROR
- *              - LZMA_DATA_ERROR: Limits (total_limit and uncompressed_limit)
- *                have been reached already.
- *              - LZMA_UNSUPPORTED_CHECK: options->check specfies a Check
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_UNSUPPORTED_CHECK: block->check specfies a Check ID
  *                that is not supported by this buid of liblzma. Initializing
  *                the encoder failed.
  *              - LZMA_PROG_ERROR
- *
- * lzma_code() can return FIXME
  */
-extern lzma_ret lzma_block_encoder(
-		lzma_stream *strm, lzma_options_block *options);
+extern lzma_ret lzma_block_encoder(lzma_stream *strm, lzma_block *block)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Initializes decoder for .lzma Block
+ * \brief       Initialize .xz Block decoder
+ *
+ * Valid actions for lzma_code() are LZMA_RUN and LZMA_FINISH. Using
+ * LZMA_FINISH is not required. It is supported only for convenience.
  *
  * \return      - LZMA_OK: All good, continue with lzma_code().
  *              - LZMA_UNSUPPORTED_CHECK: Initialization was successful, but
- *                the given Check type is not supported, thus Check will be
+ *                the given Check ID is not supported, thus Check will be
  *                ignored.
  *              - LZMA_PROG_ERROR
  *              - LZMA_MEM_ERROR
  */
-extern lzma_ret lzma_block_decoder(
-		lzma_stream *strm, lzma_options_block *options);
+extern lzma_ret lzma_block_decoder(lzma_stream *strm, lzma_block *block)
+		lzma_attr_warn_unused_result;

src/liblzma/api/lzma/check.h

@@ -22,11 +22,11 @@
 
 
 /**
- * \brief       Type of the Check
+ * \brief       Type of the integrity check (Check ID)
  *
- * The .lzma format supports multiple types of Checks that are calculated
- * from the uncompressed data (unless it is empty; then it's calculated
- * from Block Header).
+ * The .xz format supports multiple types of checks that are calculated
+ * from the uncompressed data. They very in both speed and ability to
+ * detect errors.
  */
 typedef enum {
 	LZMA_CHECK_NONE     = 0,
@@ -43,56 +43,66 @@ typedef enum {
 		 * Size of the Check field: 4 bytes
 		 */
 
-	LZMA_CHECK_CRC64    = 3,
+	LZMA_CHECK_CRC64    = 4,
 		/**<
 		 * CRC64 using the polynomial from the ECMA-182 standard
 		 *
 		 * Size of the Check field: 8 bytes
 		 */
 
-	LZMA_CHECK_SHA256   = 5
+	LZMA_CHECK_SHA256   = 10
 		/**<
 		 * SHA-256
 		 *
 		 * Size of the Check field: 32 bytes
 		 */
-} lzma_check_type;
+} lzma_check;
 
 
 /**
  * \brief       Maximum valid Check ID
  *
- * The .lzma file format specification specifies eight Check IDs (0-7). Some
- * of them are only reserved i.e. no actual Check algorithm has been assigned.
- * Still liblzma accepts any of these eight IDs for future compatibility
- * when decoding files. If a valid but unsupported Check ID is detected,
- * liblzma indicates a warning with LZMA_UNSUPPORTED_CHECK.
- *
- * FIXME bad desc
+ * The .xz file format specification specifies 16 Check IDs (0-15). Some
+ * of them are only reserved, that is, no actual Check algorithm has been
+ * assigned. When decoding, liblzma still accepts unknown Check IDs for
+ * future compatibility. If a valid but unsupported Check ID is detected,
+ * liblzma can indicate a warning; see the flags LZMA_TELL_NO_CHECK,
+ * LZMA_TELL_UNSUPPORTED_CHECK, and LZMA_TELL_ANY_CHECK in container.h.
  */
-#define LZMA_CHECK_ID_MAX 7
+#define LZMA_CHECK_ID_MAX 15
 
 
 /**
- * \brief       Check IDs supported by this liblzma build
+ * \brief       Test if the given Check ID is supported
  *
- * If lzma_available_checks[n] is true, the Check ID n is supported by this
- * liblzma build. You can assume that LZMA_CHECK_NONE and LZMA_CHECK_CRC32
- * are always available.
+ * Returns true if the given Check ID is supported by this liblzma build.
+ * Otherwise false is returned. It is safe to call this with a value that
+ * is not in the range [0, 15]; in that case the return value is always false.
+ *
+ * You can assume that LZMA_CHECK_NONE and LZMA_CHECK_CRC32 are always
+ * supported (even if liblzma is built with limited features).
  */
-extern const lzma_bool lzma_available_checks[LZMA_CHECK_ID_MAX + 1];
+extern lzma_bool lzma_check_is_supported(lzma_check check)
+		lzma_attr_const;
 
 
 /**
- * \brief       Size of the Check field with different Check IDs
+ * \brief       Get the size of the Check field with the given Check ID
  *
  * Although not all Check IDs have a check algorithm associated, the size of
- * every Check is already frozen. This array contains the size (in bytes) of
- * the Check field with specified Check ID. The values are taken from the
- * section 2.2.2 of the .lzma file format specification:
- * { 0, 4, 4, 8, 16, 32, 32, 64 }
+ * every Check is already frozen. This function returns the size (in bytes) of
+ * the Check field with the specified Check ID. The values are:
+ * { 0, 4, 4, 4, 8, 8, 8, 16, 16, 16, 32, 32, 32, 64, 64, 64 }
+ *
+ * If the argument is not in the range [0, 15], UINT32_MAX is returned.
  */
-extern const uint32_t lzma_check_sizes[LZMA_CHECK_ID_MAX + 1];
+extern uint32_t lzma_check_size(lzma_check check) lzma_attr_const;
+
+
+/**
+ * \brief       Maximum size of a Check field
+ */
+#define LZMA_CHECK_SIZE_MAX 64
 
 
 /**
@@ -109,7 +119,8 @@ extern const uint32_t lzma_check_sizes[LZMA_CHECK_ID_MAX + 1];
  * \return      Updated CRC value, which can be passed to this function
  *              again to continue CRC calculation.
  */
-extern uint32_t lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc);
+extern uint32_t lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
+		lzma_attr_pure;
 
 
 /**
@@ -119,10 +130,21 @@ extern uint32_t lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc);
  *
  * This function is used similarly to lzma_crc32(). See its documentation.
  */
-extern uint64_t lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc);
+extern uint64_t lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
+		lzma_attr_pure;
 
 
 /*
- * SHA256 functions are currently not exported to public API.
- * Contact the author if you think it should be.
+ * SHA-256 functions are currently not exported to public API.
+ * Contact Lasse Collin if you think it should be.
  */
+
+
+/**
+ * \brief       Get the type of the integrity check
+ *
+ * This function can be called only immediatelly after lzma_code() has
+ * returned LZMA_NO_CHECK, LZMA_UNSUPPORTED_CHECK, or LZMA_GET_CHECK.
+ * Calling this function in any other situation has undefined behavior.
+ */
+extern lzma_check lzma_get_check(const lzma_stream *strm);

src/liblzma/api/lzma/container.h

@@ -0,0 +1,265 @@
+/**
+ * \file        lzma/container.h
+ * \brief       File formats
+ *
+ * \author      Copyright (C) 1999-2008 Igor Pavlov
+ * \author      Copyright (C) 2007-2008 Lasse Collin
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ */
+
+#ifndef LZMA_H_INTERNAL
+#	error Never include this file directly. Use <lzma.h> instead.
+#endif
+
+
+/************
+ * Encoding *
+ ************/
+
+/**
+ * \brief       Default compression preset
+ *
+ * It's not straightforward to recommend a default preset, because in some
+ * cases keeping the resource usage relatively low is more important that
+ * getting the maximum compression ratio.
+ */
+#define LZMA_PRESET_DEFAULT     UINT32_C(6)
+
+
+/**
+ * \brief       Mask for preset level
+ *
+ * This is useful only if you need to extract the level from the preset
+ * variable. That should be rare.
+ */
+#define LZMA_PRESET_LEVEL_MASK  UINT32_C(0x1F)
+
+
+/*
+ * Preset flags
+ *
+ * Currently only one flag is defined.
+ */
+
+/**
+ * \brief       Extreme compression preset
+ *
+ * This flag modifies the preset to make the encoding significantly slower
+ * while improving the compression ratio only marginally. This is useful
+ * when you don't mind wasting time to get as small result as possible.
+ *
+ * This flag doesn't affect the memory usage requirements of the decoder (at
+ * least not significantly). The memory usage of the encoder may be increased
+ * a little but only at the lowest preset levels (0-4 or so).
+ */
+#define LZMA_PRESET_EXTREME       (UINT32_C(1) << 31)
+
+
+/**
+ * \brief       Calculate rough memory usage of easy encoder
+ *
+ * This function is a wrapper for lzma_raw_encoder_memusage().
+ *
+ * \param       preset  Compression preset (level and possible flags)
+ */
+extern uint64_t lzma_easy_encoder_memusage(uint32_t preset)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Calculate rough decoder memory usage of a preset
+ *
+ * This function is a wrapper for lzma_raw_decoder_memusage().
+ *
+ * \param       preset  Compression preset (level and possible flags)
+ */
+extern uint64_t lzma_easy_decoder_memusage(uint32_t preset)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Initialize .xz Stream encoder using a preset number
+ *
+ * This function is intended for those who just want to use the basic features
+ * if liblzma (that is, most developers out there).
+ *
+ * \param       strm    Pointer to lzma_stream that is at least initialized
+ *                      with LZMA_STREAM_INIT.
+ * \param       preset  Compression preset to use. A preset consist of level
+ *                      number and zero or more flags. Usually flags aren't
+ *                      used, so preset is simply a number [0, 9] which match
+ *                      the options -0 .. -9 of the xz command line tool.
+ *                      Additional flags can be be set using bitwise-or with
+ *                      the preset level number, e.g. 6 | LZMA_PRESET_EXTREME.
+ * \param       check   Integrity check type to use. See check.h for available
+ *                      checks. If you are unsure, use LZMA_CHECK_CRC32.
+ *
+ * \return      - LZMA_OK: Initialization succeeded. Use lzma_code() to
+ *                encode your data.
+ *              - LZMA_MEM_ERROR: Memory allocation failed. All memory
+ *                previously allocated for *strm is now freed.
+ *              - LZMA_OPTIONS_ERROR: The given compression level is not
+ *                supported by this build of liblzma.
+ *              - LZMA_UNSUPPORTED_CHECK: The given check type is not
+ *                supported by this liblzma build.
+ *              - LZMA_PROG_ERROR: One or more of the parameters have values
+ *                that will never be valid. For example, strm == NULL.
+ *
+ * If initialization succeeds, use lzma_code() to do the actual encoding.
+ * Valid values for `action' (the second argument of lzma_code()) are
+ * LZMA_RUN, LZMA_SYNC_FLUSH, LZMA_FULL_FLUSH, and LZMA_FINISH. In future,
+ * there may be compression levels or flags that don't support LZMA_SYNC_FLUSH.
+ */
+extern lzma_ret lzma_easy_encoder(
+		lzma_stream *strm, uint32_t preset, lzma_check check)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Initialize .xz Stream encoder using a custom filter chain
+ *
+ * \param       strm    Pointer to properly prepared lzma_stream
+ * \param       filters Array of filters. This must be terminated with
+ *                      filters[n].id = LZMA_VLI_UNKNOWN. See filter.h for
+ *                      more information.
+ * \param       check   Type of the integrity check to calculate from
+ *                      uncompressed data.
+ *
+ * \return      - LZMA_OK: Initialization was successful.
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_stream_encoder(lzma_stream *strm,
+		const lzma_filter *filters, lzma_check check)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Initialize .lzma encoder (legacy file format)
+ *
+ * The .lzma format is sometimes called the LZMA_Alone format, which is the
+ * reason for the name of this function. The .lzma format supports only the
+ * LZMA1 filter. There is no support for integrity checks like CRC32.
+ *
+ * Use this function if and only if you need to create files readable by
+ * legacy LZMA tools such as LZMA Utils 4.32.x. Moving to the .xz format
+ * is strongly recommended.
+ *
+ * The valid action values for lzma_code() are LZMA_RUN and LZMA_FINISH.
+ * No kind of flushing is supported, because the file format doesn't make
+ * it possible.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_alone_encoder(
+		lzma_stream *strm, const lzma_options_lzma *options)
+		lzma_attr_warn_unused_result;
+
+
+/************
+ * Decoding *
+ ************/
+
+/**
+ * This flag makes lzma_code() return LZMA_NO_CHECK if the input stream
+ * being decoded has no integrity check. Note that when used with
+ * lzma_auto_decoder(), all .lzma files will trigger LZMA_NO_CHECK
+ * if LZMA_TELL_NO_CHECK is used.
+ */
+#define LZMA_TELL_NO_CHECK              UINT32_C(0x01)
+
+
+/**
+ * This flag makes lzma_code() return LZMA_UNSUPPORTED_CHECK if the input
+ * stream has an integrity check, but the type of the integrity check is not
+ * supported by this liblzma version or build. Such files can still be
+ * decoded, but the integrity check cannot be verified.
+ */
+#define LZMA_TELL_UNSUPPORTED_CHECK     UINT32_C(0x02)
+
+
+/**
+ * This flag makes lzma_code() return LZMA_GET_CHECK as soon as the type
+ * of the integrity check is known. The type can then be got with
+ * lzma_get_check().
+ */
+#define LZMA_TELL_ANY_CHECK             UINT32_C(0x04)
+
+
+/**
+ * This flag enables decoding of concatenated files with file formats that
+ * allow concatenating compressed files as is. From the formats currently
+ * supported by liblzma, only the .xz format allows concatenated files.
+ * Concatenated files are not allowed with the legacy .lzma format.
+ *
+ * This flag also affects the usage of the `action' argument for lzma_code().
+ * When LZMA_CONCATENATED is used, lzma_code() won't return LZMA_STREAM_END
+ * unless LZMA_FINISH is used as `action'. Thus, the application has to set
+ * LZMA_FINISH in the same way as it does when encoding.
+ *
+ * If LZMA_CONCATENATED is not used, the decoders still accept LZMA_FINISH
+ * as `action' for lzma_code(), but the usage of LZMA_FINISH isn't required.
+ */
+#define LZMA_CONCATENATED               UINT32_C(0x08)
+
+
+/**
+ * \brief       Initialize .xz Stream decoder
+ *
+ * \param       strm        Pointer to properly prepared lzma_stream
+ * \param       memlimit    Rough memory usage limit as bytes
+ *
+ * \return      - LZMA_OK: Initialization was successful.
+ *              - LZMA_MEM_ERROR: Cannot allocate memory.
+ *              - LZMA_OPTIONS_ERROR: Unsupported flags
+ */
+extern lzma_ret lzma_stream_decoder(
+		lzma_stream *strm, uint64_t memlimit, uint32_t flags)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Decode .xz Streams and .lzma files with autodetection
+ *
+ * This decoder autodetects between the .xz and .lzma file formats, and
+ * calls lzma_stream_decoder() or lzma_alone_decoder() once the type
+ * of the input file has been detected.
+ *
+ * \param       strm        Pointer to properly prepared lzma_stream
+ * \param       memlimit    Rough memory usage limit as bytes
+ * \param       flags       Bitwise-or of flags, or zero for no flags.
+ *
+ * \return      - LZMA_OK: Initialization was successful.
+ *              - LZMA_MEM_ERROR: Cannot allocate memory.
+ *              - LZMA_OPTIONS_ERROR: Unsupported flags
+ */
+extern lzma_ret lzma_auto_decoder(
+		lzma_stream *strm, uint64_t memlimit, uint32_t flags)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Initialize .lzma decoder (legacy file format)
+ *
+ * Valid `action' arguments to lzma_code() are LZMA_RUN and LZMA_FINISH.
+ * There is no need to use LZMA_FINISH, but allowing it may simplify
+ * certain types of applications.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ */
+extern lzma_ret lzma_alone_decoder(lzma_stream *strm, uint64_t memlimit)
+		lzma_attr_warn_unused_result;

src/liblzma/api/lzma/copy.h

@@ -1,29 +0,0 @@
-/**
- * \file        lzma/copy.h
- * \brief       Copy filter
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Filter ID
- *
- * Filter ID of the Copy filter. This is used as lzma_options_filter.id.
- */
-#define LZMA_FILTER_COPY        LZMA_VLI_C(0x00)

src/liblzma/api/lzma/delta.h

@@ -24,9 +24,21 @@
 /**
  * \brief       Filter ID
  *
- * Filter ID of the Delta filter. This is used as lzma_options_filter.id.
+ * Filter ID of the Delta filter. This is used as lzma_filter.id.
  */
-#define LZMA_FILTER_DELTA       LZMA_VLI_C(0x20)
+#define LZMA_FILTER_DELTA       LZMA_VLI_C(0x03)
+
+
+/**
+ * \brief       Type of the delta calculation
+ *
+ * Currently only byte-wise delta is supported. Other possible types could
+ * be, for example, delta of 16/32/64-bit little/big endian integers, but
+ * these are not currently planned since byte-wise delta is almost as good.
+ */
+typedef enum {
+	LZMA_DELTA_TYPE_BYTE
+} lzma_delta_type;
 
 
 /**
@@ -35,15 +47,35 @@
  * These options are needed by both encoder and decoder.
  */
 typedef struct {
+	/** For now, this must always be LZMA_DELTA_TYPE_BYTE. */
+	lzma_delta_type type;
+
 	/**
-	 * \brief       Delta distance as bytes
+	 * \brief       Delta distance
+	 *
+	 * With the only currently supported type, LZMA_DELTA_TYPE_BYTE,
+	 * the distance is as bytes.
 	 *
 	 * Examples:
 	 *  - 16-bit stereo audio: distance = 4 bytes
 	 *  - 24-bit RGB image data: distance = 3 bytes
 	 */
-	uint32_t distance;
-#	define LZMA_DELTA_DISTANCE_MIN 1
-#	define LZMA_DELTA_DISTANCE_MAX 256
+	uint32_t dist;
+#	define LZMA_DELTA_DIST_MIN 1
+#	define LZMA_DELTA_DIST_MAX 256
+
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. You should not touch these, because the names
+	 * of these variables may change. These are and will never be used
+	 * when type is LZMA_DELTA_TYPE_BYTE, so it is safe to leave these
+	 * uninitialized.
+	 */
+	uint32_t reserved_int1;
+	uint32_t reserved_int2;
+	uint32_t reserved_int3;
+	uint32_t reserved_int4;
+	void *reserved_ptr1;
+	void *reserved_ptr2;
 
 } lzma_options_delta;

src/liblzma/api/lzma/extra.h

@@ -1,114 +0,0 @@
-/**
- * \file        lzma/extra.h
- * \brief       Handling of Extra Records in Metadata
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/*
- * Extra Record IDs
- *
- * See the .lzma file format specification for description what each
- * Extra Record type exactly means.
- *
- * If you ever need to update .lzma files with Extra Records, note that
- * the Record IDs are divided in two categories:
- *  - Safe-to-Copy Records may be preserved as is when the
- *    Stream is modified in ways that don't change the actual
- *    uncompressed data. Examples of such operatings include
- *    recompressing and adding, modifying, or deleting unrelated
- *    Extra Records.
- *  - Unsafe-to-Copy Records should be removed (and possibly
- *    recreated) when any kind of changes are made to the Stream.
- */
-
-#define LZMA_EXTRA_PADDING      0x00
-#define LZMA_EXTRA_OPENPGP      0x01
-#define LZMA_EXTRA_FILTERS      0x02
-#define LZMA_EXTRA_COMMENT      0x03
-#define LZMA_EXTRA_CHECKS       0x04
-#define LZMA_EXTRA_FILENAME     0x05
-#define LZMA_EXTRA_MTIME        0x07
-#define LZMA_EXTRA_MTIME_HR     0x09
-#define LZMA_EXTRA_MIME_TYPE    0x0B
-#define LZMA_EXTRA_HOMEPAGE     0x0D
-
-
-/**
- * \brief       Extra Records
- *
- * The .lzma format provides a way to store custom information along
- * the actual compressed content. Information about these Records
- * are passed to and from liblzma via this linked list.
- */
-typedef struct lzma_extra_s lzma_extra;
-struct lzma_extra_s {
-	/**
-	 * \brief       Pointer to the next Extra Record
-	 *
-	 * This is NULL on the last Extra Record.
-	 */
-	lzma_extra *next;
-
-	/**
-	 * \brief       Record ID
-	 *
-	 * Extra Record IDs are divided in three categories:
-	 *   - Zero is a special case used for padding. It doesn't have
-	 *     Size of Data fields.
-	 *   - Odd IDs (1, 3, 5, ...) are Safe-to-Copy IDs.
-	 *     These can be preserved as is if the Stream is
-	 *     modified in a way that doesn't alter the actual
-	 *     uncompressed content.
-	 *   - Even IDs (2, 4, 6, ...) are Unsafe-to-Copy IDs.
-	 *     If the .lzma Stream is modified in any way,
-	 *     the Extra Records having a sensitive ID should
-	 *     be removed or updated accordingly.
-	 *
-	 * Refer to the .lzma file format specification for
-	 * the up to date list of Extra Record IDs.
-	 */
-	lzma_vli id;
-
-	/**
-	 * \brief       Size of the Record data
-	 *
-	 * In case of strings, this should not include the
-	 * trailing '\0'.
-	 */
-	size_t size;
-
-	/**
-	 * \brief       Record data
-	 *
-	 * Record data is often a string in UTF-8 encoding,
-	 * but it can be arbitrary binary data. In case of
-	 * strings, the trailing '\0' is usually not stored
-	 * in the .lzma file.
-	 *
-	 * To ease working with Extra Records containing strings,
-	 * liblzma always adds '\0' to the end of data even when
-	 * it wasn't present in the .lzma file. This '\0' is not
-	 * counted in the size of the data.
-	 */
-	uint8_t *data;
-};
-
-
-extern void lzma_extra_free(lzma_extra *extra, lzma_allocator *allocator);

src/liblzma/api/lzma/filter.h

@@ -21,11 +21,28 @@
 #endif
 
 
+/**
+ * \brief       Maximum number of filters in a chain
+ *
+ * A filter chain can have 1-4 filters, of which three are allowed to change
+ * the size of the data. Usually only one or two filters are needed.
+ */
+#define LZMA_FILTERS_MAX 4
+
+
 /**
  * \brief       Filter options
  *
  * This structure is used to pass Filter ID and a pointer filter's options
- * to liblzma.
+ * to liblzma. An array of lzma_filter structures is used to define a filter
+ * chain.
+ *
+ * A filter chain is indicated with an array of lzma_filter structures.
+ * The array is terminated with .id = LZMA_VLI_UNKNOWN. Thus, the filter array
+ * must have LZMA_FILTERS_MAX + 1 elements (that is, five) to be able to hold
+ * any arbitrary filter chain. This is important when using
+ * lzma_block_header_decode() from block.h, because too small array would
+ * make liblzma write past the end of the filters array.
  */
 typedef struct {
 	/**
@@ -33,7 +50,7 @@ typedef struct {
 	 *
 	 * Use constants whose name begin with `LZMA_FILTER_' to specify
 	 * different filters. In an array of lzma_option_filter structures,
-	 * use LZMA_VLI_VALUE_UNKNOWN to indicate end of filters.
+	 * use LZMA_VLI_UNKNOWN to indicate end of filters.
 	 */
 	lzma_vli id;
 
@@ -41,7 +58,7 @@ typedef struct {
 	 * \brief       Pointer to filter-specific options structure
 	 *
 	 * If the filter doesn't need options, set this to NULL. If id is
-	 * set to LZMA_VLI_VALUE_UNKNOWN, options is ignored, and thus
+	 * set to LZMA_VLI_UNKNOWN, options is ignored, and thus
 	 * doesn't need be initialized.
 	 *
 	 * Some filters support changing the options in the middle of
@@ -51,55 +68,171 @@ typedef struct {
 	 */
 	void *options;
 
-} lzma_options_filter;
+} lzma_filter;
 
 
 /**
- * \brief       Filters available for encoding
+ * \brief       Test if the given Filter ID is supported for encoding
  *
- * Pointer to an array containing the list of available Filter IDs that
- * can be used for encoding. The last element is LZMA_VLI_VALUE_UNKNOWN.
+ * Returns true if the give Filter ID  is supported for encoding by this
+ * liblzma build. Otherwise false is returned.
  *
- * If lzma_available_filter_encoders[0] == LZMA_VLI_VALUE_UNKNOWN, the
- * encoder components haven't been built at all. This means that the
- * encoding-specific functions are probably missing from the library
- * API/ABI completely.
+ * There is no way to list which filters are available in this particular
+ * liblzma version and build. It would be useless, because the application
+ * couldn't know what kind of options the filter would need.
  */
-extern const lzma_vli *const lzma_available_filter_encoders;
+extern lzma_bool lzma_filter_encoder_is_supported(lzma_vli id);
 
 
 /**
- * \brief       Filters available for decoding
+ * \brief       Test if the given Filter ID is supported for decoding
  *
- * Pointer to an array containing the list of available Filter IDs that
- * can be used for decoding. The last element is LZMA_VLI_VALUE_UNKNOWN.
- *
- * If lzma_available_filter_decoders[0] == LZMA_VLI_VALUE_UNKNOWN, the
- * decoder components haven't been built at all. This means that the
- * decoding-specific functions are probably missing from the library
- * API/ABI completely.
+ * Returns true if the give Filter ID  is supported for decoding by this
+ * liblzma build. Otherwise false is returned.
  */
-extern const lzma_vli *const lzma_available_filter_decoders;
+extern lzma_bool lzma_filter_decoder_is_supported(lzma_vli id);
 
 
 /**
- * \brief       Calculate rough memory requirements for given filter chain
+ * \brief       Calculate rough memory requirements for raw encoder
  *
  * \param       filters     Array of filters terminated with
- *                          .id == LZMA_VLI_VALUE_UNKNOWN.
- * \param       is_encoder  Set to true when calculating memory requirements
- *                          of an encoder; false for decoder.
+ *                          .id == LZMA_VLI_UNKNOWN.
  *
- * \return      Number of mebibytes (MiB i.e. 2^20) required for the given
- *              encoder or decoder filter chain.
- *
- * \note        If calculating memory requirements of encoder, lzma_init() or
- *              lzma_init_encoder() must have been called earlier. Similarly,
- *              if calculating memory requirements of decoder, lzma_init() or
- *              lzma_init_decoder() must have been called earlier.
+ * \return      Rough number of bytes required for the given filter chain
+ *              when encoding.
  */
-extern uint32_t lzma_memory_usage(
-		const lzma_options_filter *filters, lzma_bool is_encoder);
+extern uint64_t lzma_memusage_encoder(const lzma_filter *filters)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Calculate rough memory requirements for raw decoder
+ *
+ * \param       filters     Array of filters terminated with
+ *                          .id == LZMA_VLI_UNKNOWN.
+ *
+ * \return      Rough number of bytes required for the given filter chain
+ *              when decoding.
+ */
+extern uint64_t lzma_memusage_decoder(const lzma_filter *filters)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Initialize raw encoder
+ *
+ * This function may be useful when implementing custom file formats.
+ *
+ * \param       strm    Pointer to properly prepared lzma_stream
+ * \param       options Array of lzma_filter structures.
+ *                      The end of the array must be marked with
+ *                      .id = LZMA_VLI_UNKNOWN. The minimum
+ *                      number of filters is one and the maximum is four.
+ *
+ * The `action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
+ * filter chain supports it), or LZMA_FINISH.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_raw_encoder(
+		lzma_stream *strm, const lzma_filter *options)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Initialize raw decoder
+ *
+ * The initialization of raw decoder goes similarly to raw encoder.
+ *
+ * The `action' with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using
+ * LZMA_FINISH is not required, it is supported just for convenience.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_raw_decoder(
+		lzma_stream *strm, const lzma_filter *options)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Get the size of the Filter Properties field
+ *
+ * This function may be useful when implementing custom file formats
+ * using the raw encoder and decoder.
+ *
+ * \param       size    Pointer to uint32_t to hold the size of the properties
+ * \param       filter  Filter ID and options (the size of the propeties may
+ *                      vary depending on the options)
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ *
+ * \note        This function validates the Filter ID, but does not
+ *              necessarily validate the options. Thus, it is possible
+ *              that this returns LZMA_OK while the following call to
+ *              lzma_properties_encode() returns LZMA_OPTIONS_ERROR.
+ */
+extern lzma_ret lzma_properties_size(
+		uint32_t *size, const lzma_filter *filter);
+
+
+/**
+ * \brief       Encode the Filter Properties field
+ *
+ * \param       filter  Filter ID and options
+ * \param       props   Buffer to hold the encoded options. The size of
+ *                      buffer must have been already determined with
+ *                      lzma_properties_size().
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_PROG_ERROR
+ *
+ * \note        Even this function won't validate more options than actually
+ *              necessary. Thus, it is possible that encoding the properties
+ *              succeeds but using the same options to initialize the encoder
+ *              will fail.
+ *
+ * \note        It is OK to skip calling this function if
+ *              lzma_properties_size() indicated that the size
+ *              of the Filter Properties field is zero.
+ */
+extern lzma_ret lzma_properties_encode(
+		const lzma_filter *filter, uint8_t *props);
+
+
+/**
+ * \brief       Decode the Filter Properties field
+ *
+ * \param       filter      filter->id must have been set to the correct
+ *                          Filter ID. filter->options doesn't need to be
+ *                          initialized (it's not freed by this function). The
+ *                          decoded options will be stored to filter->options.
+ *                          filter->options is set to NULL if there are no
+ *                          properties or if an error occurs.
+ * \param       allocator   Custom memory allocator used to allocate the
+ *                          options. Set to NULL to use the default malloc(),
+ *                          and in case of an error, also free().
+ * \param       props       Input buffer containing the properties.
+ * \param       props_size  Size of the properties. This must be the exact
+ *                          size; giving too much or too little input will
+ *                          return LZMA_OPTIONS_ERROR.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_OPTIONS_ERROR
+ *              - LZMA_MEM_ERROR
+ */
+extern lzma_ret lzma_properties_decode(
+		lzma_filter *filter, lzma_allocator *allocator,
+		const uint8_t *props, size_t props_size);
 
 
 /**
@@ -115,14 +248,15 @@ extern uint32_t lzma_memory_usage(
  * \return      - LZMA_OK: *size set successfully. Note that this doesn't
  *                guarantee that options->options is valid, thus
  *                lzma_filter_flags_encode() may still fail.
- *              - LZMA_HEADER_ERROR: Unknown Filter ID or unsupported options.
+ *              - LZMA_OPTIONS_ERROR: Unknown Filter ID or unsupported options.
  *              - LZMA_PROG_ERROR: Invalid options
  *
  * \note        If you need to calculate size of List of Filter Flags,
- *              you need to loop over every lzma_options_filter entry.
+ *              you need to loop over every lzma_filter entry.
  */
 extern lzma_ret lzma_filter_flags_size(
-		uint32_t *size, const lzma_options_filter *options);
+		uint32_t *size, const lzma_filter *options)
+		lzma_attr_warn_unused_result;
 
 
 /**
@@ -138,13 +272,14 @@ extern lzma_ret lzma_filter_flags_size(
  * \param       options     Filter options to be encoded
  *
  * \return      - LZMA_OK: Encoding was successful.
- *              - LZMA_HEADER_ERROR: Invalid or unsupported options.
+ *              - LZMA_OPTIONS_ERROR: Invalid or unsupported options.
  *              - LZMA_PROG_ERROR: Invalid options or not enough output
  *                buffer space (you should have checked it with
  *                lzma_filter_flags_size()).
  */
-extern lzma_ret lzma_filter_flags_encode(uint8_t *out, size_t *out_pos,
-		size_t out_size, const lzma_options_filter *options);
+extern lzma_ret lzma_filter_flags_encode(const lzma_filter *options,
+		uint8_t *out, size_t *out_pos, size_t out_size)
+		lzma_attr_warn_unused_result;
 
 
 /**
@@ -162,5 +297,7 @@ extern lzma_ret lzma_filter_flags_encode(uint8_t *out, size_t *out_pos,
  *              - LZMA_MEM_ERROR
  *              - LZMA_PROG_ERROR
  */
-extern lzma_ret lzma_filter_flags_decoder(
-		lzma_stream *strm, lzma_options_filter *options);
+extern lzma_ret lzma_filter_flags_decode(
+		lzma_filter *options, lzma_allocator *allocator,
+		const uint8_t *in, size_t *in_pos, size_t in_size)
+		lzma_attr_warn_unused_result;

src/liblzma/api/lzma/index.h

@@ -1,6 +1,6 @@
 /**
  * \file        lzma/index.h
- * \brief       Handling of Index lists
+ * \brief       Handling of .xz Index lists
  *
  * \author      Copyright (C) 1999-2006 Igor Pavlov
  * \author      Copyright (C) 2007 Lasse Collin
@@ -22,63 +22,302 @@
 
 
 /**
- * \brief
- *
- * FIXME desc
+ * \brief       Opaque data type to hold the Index
  */
 typedef struct lzma_index_s lzma_index;
-struct lzma_index_s {
+
+
+/**
+ * \brief       Index Record and its location
+ */
+typedef struct {
 	/**
-	 * \brief       Total Size of the Block
+	 * \brief       Total encoded size of a Block including Block Padding
 	 *
-	 * This includes Block Header, Compressed Data, and Block Footer.
+	 * This value is useful if you need to know the actual size of the
+	 * Block that the Block decoder will read.
 	 */
 	lzma_vli total_size;
 
 	/**
-	 * \brief       Uncompressed Size of the Block
+	 * \brief       Encoded size of a Block excluding Block Padding
+	 *
+	 * This value is stored in the Index. When doing random-access
+	 * reading, you should give this value to the Block decoder along
+	 * with uncompressed_size.
+	 */
+	lzma_vli unpadded_size;
+
+	/**
+	 * \brief       Uncompressed Size of a Block
 	 */
 	lzma_vli uncompressed_size;
 
 	/**
-	 * \brief       Pointer to the next Index Record
-	 *
-	 * This is NULL on the last Index Record.
+	 * Offset of the first byte of a Block relative to the beginning
+	 * of the Stream, or if there are multiple Indexes combined,
+	 * relative to the beginning of the first Stream.
 	 */
-	lzma_index *next;
-};
+	lzma_vli stream_offset;
+
+	/**
+	 * Uncompressed offset
+	 */
+	lzma_vli uncompressed_offset;
+
+} lzma_index_record;
+
+
+/**
+ * \brief       Calculate memory usage for Index with given number of Records
+ *
+ * On disk, the size of the Index field depends on both the number of Records
+ * stored and how big values the Records store (due to variable-length integer
+ * encoding). When the Index is kept in lzma_index structure, the memory usage
+ * depends only on the number of Records stored in the Index. The size in RAM
+ * is almost always a lot bigger than in encoded form on disk.
+ *
+ * This function calculates an approximate amount of memory needed hold the
+ * given number of Records in lzma_index structure. This value may vary
+ * between liblzma versions if the internal implementation is modified.
+ *
+ * If you want to know how much memory an existing lzma_index structure is
+ * using, use lzma_index_memusage(lzma_index_count(i)).
+ */
+extern uint64_t lzma_index_memusage(lzma_vli record_count);
+
+
+/**
+ * \brief       Allocate and initialize a new lzma_index structure
+ *
+ * If i is NULL, a new lzma_index structure is allocated, initialized,
+ * and a pointer to it returned. If allocation fails, NULL is returned.
+ *
+ * If i is non-NULL, it is reinitialized and the same pointer returned.
+ * In this case, return value cannot be NULL or a different pointer than
+ * the i that was given as an argument.
+ */
+extern lzma_index *lzma_index_init(lzma_index *i, lzma_allocator *allocator)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Deallocate the Index
+ *
+ * If i is NULL, this does nothing.
+ */
+extern void lzma_index_end(lzma_index *i, lzma_allocator *allocator);
+
+
+/**
+ * \brief       Add a new Record to an Index
+ *
+ * \param       i                 Pointer to a lzma_index structure
+ * \param       allocator         Pointer to lzma_allocator, or NULL to
+ *                                use malloc()
+ * \param       unpadded_size     Unpadded Size of a Block. This can be
+ *                                calculated with lzma_block_unpadded_size()
+ *                                after encoding or decoding the Block.
+ * \param       uncompressed_size Uncompressed Size of a Block. This can be
+ *                                taken directly from lzma_block structure
+ *                                after encoding or decoding the Block.
+ *
+ * Appending a new Record does not affect the read position.
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_DATA_ERROR: Compressed or uncompressed size of the
+ *                Stream or size of the Index field would grow too big.
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_index_append(lzma_index *i, lzma_allocator *allocator,
+		lzma_vli unpadded_size, lzma_vli uncompressed_size)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Get the number of Records
+ */
+extern lzma_vli lzma_index_count(const lzma_index *i) lzma_attr_pure;
+
+
+/**
+ * \brief       Get the size of the Index field as bytes
+ *
+ * This is needed to verify the Backward Size field in the Stream Footer.
+ */
+extern lzma_vli lzma_index_size(const lzma_index *i) lzma_attr_pure;
+
+
+/**
+ * \brief       Get the total size of the Blocks
+ *
+ * This doesn't include the Stream Header, Stream Footer, Stream Padding,
+ * or Index fields.
+ */
+extern lzma_vli lzma_index_total_size(const lzma_index *i) lzma_attr_pure;
+
+
+/**
+ * \brief       Get the total size of the Stream
+ *
+ * If multiple Indexes have been combined, this works as if the Blocks
+ * were in a single Stream.
+ */
+extern lzma_vli lzma_index_stream_size(const lzma_index *i) lzma_attr_pure;
+
+
+/**
+ * \brief       Get the total size of the file
+ *
+ * When no Indexes have been combined with lzma_index_cat(), this function is
+ * identical to lzma_index_stream_size(). If multiple Indexes have been
+ * combined, this includes also the headers of each separate Stream and the
+ * possible Stream Padding fields.
+ */
+extern lzma_vli lzma_index_file_size(const lzma_index *i) lzma_attr_pure;
+
+
+/**
+ * \brief       Get the uncompressed size of the Stream
+ */
+extern lzma_vli lzma_index_uncompressed_size(const lzma_index *i)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Get the next Record from the Index
+ */
+extern lzma_bool lzma_index_read(lzma_index *i, lzma_index_record *record)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Rewind the Index
+ *
+ * Rewind the Index so that next call to lzma_index_read() will return the
+ * first Record.
+ */
+extern void lzma_index_rewind(lzma_index *i);
+
+
+/**
+ * \brief       Locate a Record
+ *
+ * When the Index is available, it is possible to do random-access reading
+ * with granularity of Block size.
+ *
+ * \param       i       Pointer to lzma_index structure
+ * \param       record  Pointer to a structure to hold the search results
+ * \param       target  Uncompressed target offset which the caller would
+ *                      like to locate from the Stream
+ *
+ * If the target is smaller than the uncompressed size of the Stream (can be
+ * checked with lzma_index_uncompressed_size()):
+ *  - Information about the Record containing the requested uncompressed
+ *    offset is stored into *record.
+ *  - Read offset will be adjusted so that calling lzma_index_read() can be
+ *    used to read subsequent Records.
+ *  - This function returns false.
+ *
+ * If target is greater than the uncompressed size of the Stream, *record
+ * and the read position are not modified, and this function returns true.
+ */
+extern lzma_bool lzma_index_locate(
+		lzma_index *i, lzma_index_record *record, lzma_vli target)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Concatenate Indexes of two Streams
+ *
+ *
+ *
+ * \param       dest      Destination Index after which src is appended
+ * \param       src       Source Index. The memory allocated for this is
+ *                        either moved to be part of *dest or freed if and
+ *                        only if the function call succeeds, and src will
+ *                        be an invalid pointer.
+ * \param       allocator Custom memory allocator; can be NULL to use
+ *                        malloc() and free().
+ * \param       padding   Size of the Stream Padding field between Streams.
+ *                        This must be a multiple of four.
+ *
+ * \return      - LZMA_OK: Indexes concatenated successfully.
+ *              - LZMA_DATA_ERROR: *dest would grow too big.
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_index_cat(lzma_index *lzma_restrict dest,
+		lzma_index *lzma_restrict src,
+		lzma_allocator *allocator, lzma_vli padding)
+		lzma_attr_warn_unused_result;
 
 
 /**
  * \brief       Duplicates an Index list
  *
- * \return      A copy of the Index list, or NULL if memory allocation
- *              failed or the original Index was empty.
+ * Makes an identical copy of the Index. Also the read position is copied.
+ *
+ * \return      A copy of the Index, or NULL if memory allocation failed.
  */
 extern lzma_index *lzma_index_dup(
-		const lzma_index *index, lzma_allocator *allocator);
-
-
-/**
- * \brief       Frees an Index list
- *
- * All Index Recors in the list are freed. This function is convenient when
- * getting rid of lzma_metadata structures containing an Index.
- */
-extern void lzma_index_free(lzma_index *index, lzma_allocator *allocator);
-
-
-/**
- * \brief       Calculates information about the Index
- *
- * \return      LZMA_OK on success, LZMA_PROG_ERROR on error. FIXME
- */
-extern lzma_ret lzma_index_count(const lzma_index *index, size_t *count,
-		lzma_vli *lzma_restrict total_size,
-		lzma_vli *lzma_restrict uncompressed_size);
+		const lzma_index *i, lzma_allocator *allocator)
+		lzma_attr_warn_unused_result;
 
 
 /**
  * \brief       Compares if two Index lists are identical
+ *
+ * \return      True if *a and *b are equal, false otherwise.
  */
-extern lzma_bool lzma_index_is_equal(const lzma_index *a, const lzma_index *b);
+extern lzma_bool lzma_index_equal(const lzma_index *a, const lzma_index *b)
+		lzma_attr_pure;
+
+
+/**
+ * \brief       Initializes Index encoder
+ *
+ * \param       strm        Pointer to properly prepared lzma_stream
+ * \param       i           Pointer to lzma_index which should be encoded.
+ *                          The read position will be at the end of the Index
+ *                          after lzma_code() has returned LZMA_STREAM_END.
+ *
+ * The only valid action value for lzma_code() is LZMA_RUN.
+ *
+ * \return      - LZMA_OK: Initialization succeeded, continue with lzma_code().
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_index_encoder(lzma_stream *strm, lzma_index *i)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Initializes Index decoder
+ *
+ * \param       strm        Pointer to properly prepared lzma_stream
+ * \param       i           Pointer to a pointer that will be made to point
+ *                          to the final decoded Index once lzma_code() has
+ *                          returned LZMA_STREAM_END. That is,
+ *                          lzma_index_decoder() takes care of allocating
+ *                          a new lzma_index structure.
+ * \param       memlimit    How much memory the resulting Index is allowed
+ *                          to require.
+ *
+ * The only valid action value for lzma_code() is LZMA_RUN.
+ *
+ * \return      - LZMA_OK: Initialization succeeded, continue with lzma_code().
+ *              - LZMA_MEM_ERROR
+ *              - LZMA_MEMLIMIT_ERROR
+ *              - LZMA_PROG_ERROR
+ *
+ * \note        The memory usage limit is checked early in the decoding
+ *              (within the first dozen input bytes or so). The actual memory
+ *              is allocated later in smaller pieces. If the memory usage
+ *              limit is modified after decoding a part of the Index already,
+ *              the new limit may be ignored.
+ */
+extern lzma_ret lzma_index_decoder(
+		lzma_stream *strm, lzma_index **i, uint64_t memlimit)
+		lzma_attr_warn_unused_result;

src/liblzma/api/lzma/index_hash.h

@@ -0,0 +1,110 @@
+/**
+ * \file        lzma/index_hash.h
+ * \brief       Validates Index by using a hash function
+ *
+ * Instead of constructing complete Index while decoding Blocks, Index hash
+ * calculates a hash of the Block sizes and Index, and then compares the
+ * hashes. This way memory usage is constant even with large number of
+ * Blocks and huge Index.
+ *
+ * \author      Copyright (C) 2008 Lasse Collin
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ */
+
+#ifndef LZMA_H_INTERNAL
+#	error Never include this file directly. Use <lzma.h> instead.
+#endif
+
+/**
+ * \brief       Opaque data type to hold the Index hash
+ */
+typedef struct lzma_index_hash_s lzma_index_hash;
+
+
+/**
+ * \brief       Allocate and initialize a new lzma_index_hash structure
+ *
+ * If index_hash is NULL, a new lzma_index_hash structure is allocated,
+ * initialized, and a pointer to it returned. If allocation fails, NULL
+ * is returned.
+ *
+ * If index_hash is non-NULL, it is reinitialized and the same pointer
+ * returned. In this case, return value cannot be NULL or a different
+ * pointer than the index_hash that was given as an argument.
+ */
+extern lzma_index_hash *lzma_index_hash_init(
+		lzma_index_hash *index_hash, lzma_allocator *allocator)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Deallocate lzma_index_hash structure
+ */
+extern void lzma_index_hash_end(
+		lzma_index_hash *index_hash, lzma_allocator *allocator);
+
+
+/**
+ * \brief       Add a new Record to an Index hash
+ *
+ * \param       index             Pointer to a lzma_index_hash structure
+ * \param       unpadded_size     Unpadded Size of a Block
+ * \param       uncompressed_size Uncompressed Size of a Block
+ *
+ * \return      - LZMA_OK
+ *              - LZMA_DATA_ERROR: Compressed or uncompressed size of the
+ *                Stream or size of the Index field would grow too big.
+ *              - LZMA_PROG_ERROR: Invalid arguments or this function is being
+ *                used when lzma_index_hash_decode() has already been used.
+ */
+extern lzma_ret lzma_index_hash_append(lzma_index_hash *index_hash,
+		lzma_vli unpadded_size, lzma_vli uncompressed_size)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Decode and validate the Index field
+ *
+ * After telling the sizes of all Blocks with lzma_index_hash_append(),
+ * the actual Index field is decoded with this function. Specifically,
+ * once decoding of the Index field has been started, no more Records
+ * can be added using lzma_index_hash_append().
+ *
+ * This function doesn't use lzma_stream structure to pass the input data.
+ * Instead, the input buffer is specified using three arguments. This is
+ * because it matches better the internal APIs of liblzma.
+ *
+ * \param       index_hash      Pointer to a lzma_index_hash structure
+ * \param       in              Pointer to the beginning of the input buffer
+ * \param       in_pos          in[*in_pos] is the next byte to process
+ * \param       in_size         in[in_size] is the first byte not to process
+ *
+ * \return      - LZMA_OK: So far good, but more input is needed.
+ *              - LZMA_STREAM_END: Index decoded successfully and it matches
+ *                the Records given with lzma_index_hash_append().
+ *              - LZMA_DATA_ERROR: Index is corrupt or doesn't match the
+ *                information given with lzma_index_hash_append().
+ *              - LZMA_BUF_ERROR: Cannot progress because *in_pos >= in_size.
+ *              - LZMA_PROG_ERROR
+ */
+extern lzma_ret lzma_index_hash_decode(lzma_index_hash *index_hash,
+		const uint8_t *in, size_t *in_pos, size_t in_size)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Get the size of the Index field as bytes
+ *
+ * This is needed to verify the Backward Size field in the Stream Footer.
+ */
+extern lzma_vli lzma_index_hash_size(const lzma_index_hash *index_hash)
+		lzma_attr_pure;

src/liblzma/api/lzma/info.h

@@ -1,315 +0,0 @@
-/**
- * \file        lzma/info.h
- * \brief       Handling of Stream size information
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**********
- * Basics *
- **********/
-
-/**
- * \brief       Opaque data type to hold the size information
- */
-typedef struct lzma_info_s lzma_info;
-
-
-typedef struct {
-	/**
-	 * \brief       Total Size of this Block
-	 *
-	 * This can be LZMA_VLI_VALUE_UNKNOWN.
-	 */
-	lzma_vli total_size;
-
-	/**
-	 * \brief       Uncompressed Size of this Block
-	 *
-	 * This can be LZMA_VLI_VALUE_UNKNOWN.
-	 */
-	lzma_vli uncompressed_size;
-
-	/**
-	 * \brief       Offset of the first byte of the Block
-	 *
-	 * In encoder, this is useful to find out the alignment of the Block.
-	 *
-	 * In decoder, this is useful when doing random-access reading
-	 * with help from lzma_info_data_locate().
-	 */
-	lzma_vli stream_offset;
-
-	/**
-	 * \brief       Uncompressed offset of the Block
-	 *
-	 * Offset of the first uncompressed byte of the Block relative to
-	 * all uncompressed data in the Block.
-	 * FIXME desc
-	 */
-	lzma_vli uncompressed_offset;
-
-	/**
-	 * \brief       Pointers to internal data structures
-	 *
-	 * Applications must not touch these.
-	 */
-	void *internal[4];
-
-} lzma_info_iter;
-
-
-typedef enum {
-	LZMA_INFO_STREAM_START,
-	LZMA_INFO_HEADER_METADATA,
-	LZMA_INFO_TOTAL,
-	LZMA_INFO_UNCOMPRESSED,
-	LZMA_INFO_FOOTER_METADATA
-} lzma_info_size;
-
-
-/**
- * \brief       Allocates and initializes a new lzma_info structure
- *
- * If info is NULL, a new lzma_info structure is allocated, initialized, and
- * a pointer to it returned. If allocation fails, NULL is returned.
- *
- * If info is non-NULL, it is reinitialized and the same pointer returned.
- * (In this case, return value cannot be NULL or a different pointer than
- * the info given as argument.)
- */
-extern lzma_info *lzma_info_init(lzma_info *info, lzma_allocator *allocator);
-
-
-/**
- * \brief       Resets lzma_info
- *
- * This is like calling lzma_info_end() and lzma_info_create(), but
- * re-uses the existing base structure.
- */
-extern void lzma_info_reset(
-		lzma_info *info, lzma_allocator *allocator);
-
-
-/**
- * \brief       Frees memory allocated for a lzma_info structure
- */
-extern void lzma_info_free(lzma_info *info, lzma_allocator *allocator);
-
-
-/************************
- * Setting known values *
- ************************/
-
-/**
- * \brief       Set a known size value
- *
- * \param       info    Pointer returned by lzma_info_create()
- * \param       type    Any value from lzma_info_size
- * \param       size    Value to set or verify
- *
- * \return      LZMA_OK on success, LZMA_DATA_ERROR if the size doesn't
- *              match the existing information, or LZMA_PROG_ERROR
- *              if type is invalid or size is not a valid VLI.
- */
-extern lzma_ret lzma_info_size_set(
-		lzma_info *info, lzma_info_size type, lzma_vli size);
-
-
-/**
- * \brief       Sets the Index
- *
- * The given lzma_index list is "absorbed" by this function. The application
- * must not access it after this function call, even if this function returns
- * an error.
- *
- * \note        The given lzma_index will at some point get freed by the
- *              lzma_info_* functions. If you use a custom lzma_allocator,
- *              make sure that it can free the lzma_index.
- */
-extern lzma_ret lzma_info_index_set(
-		lzma_info *info, lzma_allocator *allocator,
-		lzma_index *index, lzma_bool eat_index);
-
-
-/**
- * \brief       Sets information from a known Metadata Block
- *
- * This is a shortcut for calling lzma_info_size_set() with different type
- * arguments, lzma_info_index_set() with metadata->index.
- */
-extern lzma_ret lzma_info_metadata_set(lzma_info *info,
-		lzma_allocator *allocator, lzma_metadata *metadata,
-		lzma_bool is_header_metadata, lzma_bool eat_index);
-
-
-/***************
- * Incremental *
- ***************/
-
-/**
- * \brief       Prepares an iterator to be used with given lzma_info structure
- *
- *
- */
-extern void lzma_info_iter_begin(lzma_info *info, lzma_info_iter *iter);
-
-
-/**
- * \brief       Moves to the next Index Record
- *
- *
- */
-extern lzma_ret lzma_info_iter_next(
-		lzma_info_iter *iter, lzma_allocator *allocator);
-
-
-/**
- * \brief       Sets or verifies the sizes in the Index Record
- *
- * \param       iter    Pointer to iterator to be set or verified
- * \param       total_size
- *                      Total Size in bytes or LZMA_VLI_VALUE_UNKNOWN
- * \param       uncompressed_size
- *                      Uncompressed Size or LZMA_VLI_VALUE_UNKNOWN
- *
- * \return      - LZMA_OK: All OK.
- *              - LZMA_DATA_ERROR: Given sizes don't match with the already
- *                known sizes.
- *              - LZMA_PROG_ERROR: Internal error, possibly integer
- *                overflow (e.g. the sum of all the known sizes is too big)
- */
-extern lzma_ret lzma_info_iter_set(lzma_info_iter *iter,
-		lzma_vli total_size, lzma_vli uncompressed_size);
-
-
-/**
- * \brief       Locates a Data Block
- *
- * \param       iter        Properly initialized iterator
- * \param       allocator   Pointer to lzma_allocator or NULL
- * \param       uncompressed_offset
- *                          Target offset to locate. The final offset
- *                          will be equal or smaller than this.
- * \param       allow_alloc True if this function is allowed to call
- *                          lzma_info_iter_next() to allocate a new Record
- *                          if the requested offset reached end of Index
- *                          Record list. Note that if Index has been marked
- *                          final, lzma_info_iter_next() is never called.
- *
- * \return      - LZMA_OK: All OK, *iter updated accordingly.
- *              - LZMA_DATA_ERROR: Trying to search past the end of the Index
- *                Record list, and allocating a new Record was not allowed
- *                either because allow_alloc was false or Index was final.
- *              - LZMA_PROG_ERROR: Internal error (probably integer
- *                overflow causing some lzma_vli getting too big).
- */
-extern lzma_ret lzma_info_iter_locate(lzma_info_iter *iter,
-		lzma_allocator *allocator, lzma_vli uncompressed_offset,
-		lzma_bool allow_alloc);
-
-
-/**
- * \brief       Finishes incrementally constructed Index
- *
- * This sets the known Total Size and Uncompressed of the Data Blocks
- * based on the information collected from the Index Records, and marks
- * the Index as final.
- */
-extern lzma_ret lzma_info_index_finish(lzma_info *info);
-
-
-/***************************
- * Reading the information *
- ***************************/
-
-/**
- * \brief       Gets a known size
- *
- *
- */
-extern lzma_vli lzma_info_size_get(
-		const lzma_info *info, lzma_info_size type);
-
-extern lzma_vli lzma_info_metadata_locate(
-		const lzma_info *info, lzma_bool is_header_metadata);
-
-/**
- * \brief       Gets a pointer to the beginning of the Index list
- *
- * If detach is true, the Index will be detached from the lzma_info
- * structure, and thus not be modified or freed by lzma_info_end().
- *
- * If detach is false, the application must not modify the Index in any way.
- * Also, the Index list is guaranteed to be valid only till the next call
- * to any lzma_info_* function.
- */
-extern lzma_index *lzma_info_index_get(lzma_info *info, lzma_bool detach);
-
-
-extern size_t lzma_info_index_count_get(const lzma_info *info);
-
-
-extern uint32_t lzma_info_metadata_alignment_get(
-		const lzma_info *info, lzma_bool is_header_metadata);
-
-
-
-/**
- * \brief       Locate a Block containing the given uncompressed offset
- *
- * This function is useful when you need to do random-access reading in
- * a Multi-Block Stream.
- *
- * \param       info    Pointer to lzma_info that has at least one
- *                      Index Record. The Index doesn't need to be finished.
- * \param       uncompressed_target
- *                      Uncompressed target offset which the caller would
- *                      like to locate from the Stream.
- * \param       stream_offset
- *                      Starting offset (relative to the beginning the Stream)
- *                      of the Block containing the requested location.
- * \param       uncompressed_offset
- *                      The actual uncompressed offset of the beginning of
- *                      the Block. uncompressed_offset <= uncompressed_target
- *                      is always true; the application needs to uncompress
- *                      uncompressed_target - uncompressed_offset bytes to
- *                      reach the requested target offset.
- * \param       total_size
- *                      Total Size of the Block. If the Index is incomplete,
- *                      this may be LZMA_VLI_VALUE_UNKNOWN indicating unknown
- *                      size.
- * \param       uncompressed_size
- *                      Uncompressed Size of the Block. If the Index is
- *                      incomplete, this may be LZMA_VLI_VALUE_UNKNOWN
- *                      indicating unknown size. The application must pass
- *                      this value to the Block decoder to verify FIXME
- *
- * \return
- *
- * \note        This function is currently implemented as a linear search.
- *              If there are many Index Records, this can be really slow.
- *              This can be improved in newer liblzma versions if needed.
- */
-extern lzma_bool lzma_info_data_locate(const lzma_info *info,
-		lzma_vli uncompressed_target,
-		lzma_vli *lzma_restrict stream_offset,
-		lzma_vli *lzma_restrict uncompressed_offset,
-		lzma_vli *lzma_restrict total_size,
-		lzma_vli *lzma_restrict uncompressed_size);

src/liblzma/api/lzma/init.h

@@ -1,85 +0,0 @@
-/**
- * \file        lzma/init.h
- * \brief       Initializations
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Initialize all internal static variables
- *
- * Depending on the build options, liblzma may have some internal static
- * variables, that must be initialized before using any other part of
- * the library (*). It is recommended to do these initializations in the very
- * beginning of the application by calling appropriate initialization function.
- *
- * (*) There are some exceptions to this rule. FIXME
- *
- * The initialization functions are not necessarily thread-safe, thus the
- * required initializations must be done before creating any threads. (The
- * rest of the functions of liblzma are thread-safe.) Calling the
- * initialization functions multiple times does no harm, although it
- * still shouldn't be done when there are multiple threads running.
- *
- * lzma_init() initializes all internal static variables by calling
- * lzma_lzma_init_encoder() and lzma_init_decoder().
- *
- * If you need only encoder, decoder, or neither-encoder-nor-decoder
- * functions, you may use other initialization functions, which initialize
- * only a subset of liblzma's internal static variables. Using those
- * functions have the following advantages:
- *  - When linking statically against liblzma, less useless functions will
- *    get linked into the binary. E.g. if you need only the decoder functions,
- *    using lzma_init_decoder() avoids linking bunch of encoder related code.
- *  - There is less things to initialize, making the initialization
- *    process slightly faster.
- */
-extern void lzma_init(void);
-
-
-/**
- * \brief       Initialize internal static variables needed by encoders
- *
- * If you need only the encoder functions, you may use this function to
- * initialize only the things required by encoders.
- *
- * This function also calls lzma_init_check().
- */
-extern void lzma_init_encoder(void);
-
-
-/**
- * \brief       Initialize internal static variables needed by decoders
- *
- * If you need only the decoder functions, you may use this function to
- * initialize only the things required by decoders.
- *
- * This function also calls lzma_init_check().
- */
-extern void lzma_init_decoder(void);
-
-
-/**
- * \brief       Initialize internal static variables needed by integrity checks
- *
- * Currently this initializes CRC32 and CRC64 lookup tables if precalculated
- * tables haven't been built into the library. This function can be useful
- * if the only thing you need from liblzma is the integrity check functions.
- */
-extern void lzma_init_check(void);

src/liblzma/api/lzma/lzma.h

@@ -1,6 +1,6 @@
 /**
  * \file        lzma/lzma.h
- * \brief       LZMA filter
+ * \brief       LZMA1 and LZMA2 filters
  *
  * \author      Copyright (C) 1999-2006 Igor Pavlov
  * \author      Copyright (C) 2007 Lasse Collin
@@ -22,29 +22,114 @@
 
 
 /**
- * \brief       Filter ID
+ * \brief       LZMA1 Filter ID
  *
- * Filter ID of the LZMA filter. This is used as lzma_options_filter.id.
+ * LZMA1 is the very same thing as what was called just LZMA in earlier
+ * LZMA Utils, 7-Zip, and LZMA SDK. It's called LZMA1 here to prevent
+ * developers from accidentally using LZMA when they actually want LZMA2.
  */
-#define LZMA_FILTER_LZMA        LZMA_VLI_C(0x40)
+#define LZMA_FILTER_LZMA1       LZMA_VLI_C(0x4000000000000001)
+
+/**
+ * \brief       LZMA2 Filter ID
+ *
+ * Usually you want this instead of LZMA1. Compared to LZMA1, LZMA2 adds
+ * support for LZMA_SYNC_FLUSH, uncompressed chunks (expands uncompressible
+ * data less), possibility to change lc/lp/pb in the middle of encoding, and
+ * some other internal improvements.
+ */
+#define LZMA_FILTER_LZMA2       LZMA_VLI_C(0x21)
+
+
+/**
+ * \brief       Match finders
+ *
+ * Match finder has major effect on both speed and compression ratio.
+ * Usually hash chains are faster than binary trees.
+ *
+ * The memory usage formulas are only rough estimates, which are closest to
+ * reality when dict_size is a power of two. The formulas are  more complex
+ * in reality, and can also change a little between liblzma versions. Use
+ * lzma_memusage_encoder() to get more accurate estimate of memory usage.
+ */
+typedef enum {
+	LZMA_MF_HC3     = 0x03,
+		/**<
+		 * \brief       Hash Chain with 2- and 3-byte hashing
+		 *
+		 * Minimum nice_len: 3
+		 *
+		 * Memory usage:
+		 *  - dict_size <= 16 MiB: dict_size * 7.5
+		 *  - dict_size > 16 MiB: dict_size * 5.5 + 64 MiB
+		 */
+
+	LZMA_MF_HC4     = 0x04,
+		/**<
+		 * \brief       Hash Chain with 2-, 3-, and 4-byte hashing
+		 *
+		 * Minimum nice_len: 4
+		 *
+		 * Memory usage: dict_size * 7.5
+		 */
+
+	LZMA_MF_BT2     = 0x12,
+		/**<
+		 * \brief       Binary Tree with 2-byte hashing
+		 *
+		 * Minimum nice_len: 2
+		 *
+		 * Memory usage: dict_size * 9.5
+		 */
+
+	LZMA_MF_BT3     = 0x13,
+		/**<
+		 * \brief       Binary Tree with 2- and 3-byte hashing
+		 *
+		 * Minimum nice_len: 3
+		 *
+		 * Memory usage:
+		 *  - dict_size <= 16 MiB: dict_size * 11.5
+		 *  - dict_size > 16 MiB: dict_size * 9.5 + 64 MiB
+		 */
+
+	LZMA_MF_BT4     = 0x14
+		/**<
+		 * \brief       Binary Tree with 2-, 3-, and 4-byte hashing
+		 *
+		 * Minimum nice_len: 4
+		 *
+		 * Memory usage: dict_size * 11.5
+		 */
+} lzma_match_finder;
+
+
+/**
+ * \brief       Test if given match finder is supported
+ *
+ * Returns true if the given match finder is supported by this liblzma build.
+ * Otherwise false is returned. It is safe to call this with a value that
+ * isn't listed in lzma_match_finder enumeration; the return value will be
+ * false.
+ *
+ * There is no way to list which match finders are available in this
+ * particular liblzma version and build. It would be useless, because
+ * a new match finder, which the application developer wasn't aware,
+ * could require giving additional options to the encoder that the older
+ * match finders don't need.
+ */
+extern lzma_bool lzma_mf_is_supported(lzma_match_finder match_finder)
+		lzma_attr_const;
 
 
 /**
  * \brief       LZMA compression modes
  *
- * Currently there are only two modes. Earlier LZMA SDKs had also third
- * mode between fast and best.
+ * This selects the function used to analyze the data produced by the match
+ * finder.
  */
 typedef enum {
-	LZMA_MODE_INVALID = -1,
-		/**<
-		 * \brief       Invalid mode
-		 *
-		 * Used as array terminator in lzma_available_modes.
-		 */
-
-
-	LZMA_MODE_FAST = 0,
+	LZMA_MODE_FAST = 1,
 		/**<
 		 * \brief       Fast compression
 		 *
@@ -52,9 +137,9 @@ typedef enum {
 		 * a hash chain match finder.
 		 */
 
-	LZMA_MODE_BEST = 2
+	LZMA_MODE_NORMAL = 2
 		/**<
-		 * \brief       Best compression ratio
+		 * \brief       Normal compression
 		 *
 		 * This is usually notably slower than fast mode. Use this
 		 * together with binary tree match finders to expose the
@@ -64,80 +149,31 @@ typedef enum {
 
 
 /**
- * \brief       Match finders
+ * \brief       Test if given compression mode is supported
  *
- * Match finder has major effect on both speed and compression ratio.
- * Usually hash chains are faster than binary trees.
+ * Returns true if the given compression mode is supported by this liblzma
+ * build. Otherwise false is returned. It is safe to call this with a value
+ * that isn't listed in lzma_mode enumeration; the return value will be false.
+ *
+ * There is no way to list which modes are available in this particular
+ * liblzma version and build. It would be useless, because a new compression
+ * mode, which the application developer wasn't aware, could require giving
+ * additional options to the encoder that the older modes don't need.
  */
-typedef enum {
-	LZMA_MF_INVALID = -1,
-		/**<
-		 * \brief       Invalid match finder ID
-		 *
-		 * Used as array terminator in lzma_available_match_finders.
-		 */
-
-	LZMA_MF_HC3     = 0x03,
-		/**<
-		 * \brief       Hash Chain with 3 bytes hashing
-		 *
-		 * \todo Memory requirements
-		 *
-		 * \note        It's possible that this match finder gets
-		 *              removed in future. The definition will stay
-		 *              in this header, but liblzma may return
-		 *              LZMA_HEADER_ERROR if it is specified (just
-		 *              like it would if the match finder had been
-		 *              disabled at compile time).
-		 */
-
-	LZMA_MF_HC4     = 0x04,
-		/**<
-		 * \brief       Hash Chain with 4 bytes hashing
-		 *
-		 * Memory requirements: 7.5 * dictionary_size + 4 MiB
-		 *
-		 * \note        It's possible that this match finder gets
-		 *              removed in future. The definition will stay
-		 *              in this header, but liblzma may return
-		 *              LZMA_HEADER_ERROR if it is specified (just
-		 *              like it would if the match finder had been
-		 *              disabled at compile time).
-		 */
-
-	LZMA_MF_BT2     = 0x12,
-		/**<
-		 * \brief       Binary Tree with 2 bytes hashing
-		 *
-		 * Memory requirements: 9.5 * dictionary_size + 4 MiB
-		 */
-
-	LZMA_MF_BT3     = 0x13,
-		/**<
-		 * \brief       Binary Tree with 3 bytes hashing
-		 *
-		 * Memory requirements: 11.5 * dictionary_size + 4 MiB
-		 */
-
-	LZMA_MF_BT4     = 0x14
-		/**<
-		 * \brief       Binary Tree with 4 bytes hashing
-		 *
-		 * Memory requirements: 11.5 * dictionary_size + 4 MiB
-		 */
-} lzma_match_finder;
+extern lzma_bool lzma_mode_is_supported(lzma_mode mode) lzma_attr_const;
 
 
 /**
- * \brief       Options specific to the LZMA method handler
+ * \brief       Options specific to the LZMA1 and LZMA2 filters
+ *
+ * Since LZMA1 and LZMA2 share most of the code, it's simplest to share
+ * the options structure too. For encoding, all but the reserved variables
+ * need to be initialized unless specifically mentioned otherwise.
+ *
+ * For raw decoding, both LZMA1 and LZMA2 need dict_size, preset_dict, and
+ * preset_dict_size (if preset_dict != NULL). LZMA1 needs also lc, lp, and pb.
  */
 typedef struct {
-	/**********************************
-	 * LZMA encoding/decoding options *
-	 **********************************/
-
-	/* These options are required in encoder and also with raw decoding. */
-
 	/**
 	 * \brief       Dictionary size in bytes
 	 *
@@ -147,61 +183,30 @@ typedef struct {
 	 * indicate what data to repeat from the dictionary buffer. Thus,
 	 * the bigger the dictionary, the better compression ratio usually is.
 	 *
-	 * Raw decoding: Too big dictionary does no other harm than
-	 * wasting memory. This value is ignored by lzma_raw_decode_buffer(),
-	 * because it uses the target buffer as the dictionary.
+	 * Maximum size of the dictionary depends on multiple things:
+	 *  - Memory usage limit
+	 *  - Available address space (not a problem on 64-bit systems)
+	 *  - Selected match finder (encoder only)
+	 *
+	 * Currently the maximum dictionary size for encoding is 1.5 GiB
+	 * (i.e. (UINT32_C(1) << 30) + (UINT32_C(1) << 29)) even on 64-bit
+	 * systems for certain match finder implementation reasons. In future,
+	 * there may be match finders that support bigger dictionaries (3 GiB
+	 * will probably be the maximum).
+	 *
+	 * Decoder already supports dictionaries up to 4 GiB - 1 B (i.e.
+	 * UINT32_MAX), so increasing the maximum dictionary size of the
+	 * encoder won't cause problems for old decoders.
+	 *
+	 * Because extremely small dictionaries sizes would have unneeded
+	 * overhead in the decoder, the minimum dictionary size is 4096 bytes.
+	 *
+	 * \note        When decoding, too big dictionary does no other harm
+	 *              than wasting memory.
 	 */
-	uint32_t dictionary_size;
-#	define LZMA_DICTIONARY_SIZE_MIN            1
-#	define LZMA_DICTIONARY_SIZE_MAX            (UINT32_C(1) << 30)
-#	define LZMA_DICTIONARY_SIZE_DEFAULT        (UINT32_C(1) << 23)
-
-	/**
-	 * \brief       Number of literal context bits
-	 *
-	 * How many of the highest bits of the previous uncompressed
-	 * eight-bit byte (also known as `literal') are taken into
-	 * account when predicting the bits of the next literal.
-	 *
-	 * \todo        Example
-	 */
-	uint32_t literal_context_bits;
-#	define LZMA_LITERAL_CONTEXT_BITS_MIN       0
-#	define LZMA_LITERAL_CONTEXT_BITS_MAX       8
-#	define LZMA_LITERAL_CONTEXT_BITS_DEFAULT   3
-
-	/**
-	 * \brief       Number of literal position bits
-	 *
-	 * How many of the lowest bits of the current position (number
-	 * of bytes from the beginning of the uncompressed data) in the
-	 * uncompressed data is taken into account when predicting the
-	 * bits of the next literal (a single eight-bit byte).
-	 *
-	 * \todo        Example
-	 */
-	uint32_t literal_pos_bits;
-#	define LZMA_LITERAL_POS_BITS_MIN           0
-#	define LZMA_LITERAL_POS_BITS_MAX           4
-#	define LZMA_LITERAL_POS_BITS_DEFAULT       0
-
-	/**
-	 * \brief       Number of position bits
-	 *
-	 * How many of the lowest bits of the current position in the
-	 * uncompressed data is taken into account when estimating
-	 * probabilities of matches. A match is a sequence of bytes for
-	 * which a matching sequence is found from the dictionary and
-	 * thus can be stored as distance-length pair.
-	 *
-	 * Example: If most of the matches occur at byte positions
-	 * of 8 * n + 3, that is, 3, 11, 19, ... set pos_bits to 3,
-	 * because 2**3 == 8.
-	 */
-	uint32_t pos_bits;
-#	define LZMA_POS_BITS_MIN                   0
-#	define LZMA_POS_BITS_MAX                   4
-#	define LZMA_POS_BITS_DEFAULT               2
+	uint32_t dict_size;
+#	define LZMA_DICT_SIZE_MIN       UINT32_C(4096)
+#	define LZMA_DICT_SIZE_DEFAULT   (UINT32_C(1) << 23)
 
 	/**
 	 * \brief       Pointer to an initial dictionary
@@ -228,85 +233,177 @@ typedef struct {
 	 *
 	 * \todo        This feature is not implemented yet.
 	 */
-	const uint8_t *preset_dictionary;
+	const uint8_t *preset_dict;
 
 	/**
 	 * \brief       Size of the preset dictionary
 	 *
 	 * Specifies the size of the preset dictionary. If the size is
-	 * bigger than dictionary_size, only the last dictionary_size
-	 * bytes are processed.
+	 * bigger than dict_size, only the last dict_size bytes are processed.
 	 *
-	 * This variable is read only when preset_dictionary is not NULL.
+	 * This variable is read only when preset_dict is not NULL.
 	 */
-	uint32_t preset_dictionary_size;
+	uint32_t preset_dict_size;
 
-	/******************************************
-	 * LZMA options needed only when encoding *
-	 ******************************************/
+	/**
+	 * \brief       Number of literal context bits
+	 *
+	 * How many of the highest bits of the previous uncompressed
+	 * eight-bit byte (also known as `literal') are taken into
+	 * account when predicting the bits of the next literal.
+	 *
+	 * \todo        Example
+	 *
+	 * There is a limit that applies to literal context bits and literal
+	 * position bits together: lc + lp <= 4. Without this limit the
+	 * decoding could become very slow, which could have security related
+	 * results in some cases like email servers doing virus scanning.
+	 * This limit also simplifies the internal implementation in liblzma.
+	 *
+	 * There may be LZMA streams that have lc + lp > 4 (maximum lc
+	 * possible would be 8). It is not possible to decode such streams
+	 * with liblzma.
+	 */
+	uint32_t lc;
+#	define LZMA_LCLP_MIN    0
+#	define LZMA_LCLP_MAX    4
+#	define LZMA_LC_DEFAULT  3
+
+	/**
+	 * \brief       Number of literal position bits
+	 *
+	 * How many of the lowest bits of the current position (number
+	 * of bytes from the beginning of the uncompressed data) in the
+	 * uncompressed data is taken into account when predicting the
+	 * bits of the next literal (a single eight-bit byte).
+	 *
+	 * \todo        Example
+	 */
+	uint32_t lp;
+#	define LZMA_LP_DEFAULT  0
+
+	/**
+	 * \brief       Number of position bits
+	 *
+	 * How many of the lowest bits of the current position in the
+	 * uncompressed data is taken into account when estimating
+	 * probabilities of matches. A match is a sequence of bytes for
+	 * which a matching sequence is found from the dictionary and
+	 * thus can be stored as distance-length pair.
+	 *
+	 * Example: If most of the matches occur at byte positions of
+	 * 8 * n + 3, that is, 3, 11, 19, ... set pb to 3, because 2**3 == 8.
+	 */
+	uint32_t pb;
+#	define LZMA_PB_MIN      0
+#	define LZMA_PB_MAX      4
+#	define LZMA_PB_DEFAULT  2
+
+	/**
+	 * \brief       Indicate if the options structure is persistent
+	 *
+	 * If this is true, the application must keep this options structure
+	 * available after the LZMA2 encoder has been initialized. With
+	 * persistent structure it is possible to change some encoder options
+	 * in the middle of the encoding process without resetting the encoder.
+	 *
+	 * This option is used only by LZMA2. LZMA1 ignores this and it is
+	 * safe to not initialize this when encoding with LZMA1.
+	 */
+	lzma_bool persistent;
 
 	/** LZMA compression mode */
 	lzma_mode mode;
 
 	/**
-	 * \brief       Number of fast bytes
+	 * \brief       Nice length of a match
 	 *
-	 * Number of fast bytes determines how many bytes the encoder
-	 * compares from the match candidates when looking for the best
-	 * match. Bigger fast bytes value usually increase both compression
-	 * ratio and time.
+	 * This determines how many bytes the encoder compares from the match
+	 * candidates when looking for the best match. Once a match of at
+	 * least nice_len bytes long is found, the encoder stops looking for
+	 * better condidates and encodes the match. (Naturally, if the found
+	 * match is actually longer than nice_len, the actual length is
+	 * encoded; it's not truncated to nice_len.)
+	 *
+	 * Bigger values usually increase the compression ratio and
+	 * compression time. For most files, 30 to 100 is a good value,
+	 * which gives very good compression ratio at good speed.
+	 *
+	 * The exact minimum value depends on the match finder. The maximum is
+	 * 273, which is the maximum length of a match that LZMA can encode.
 	 */
-	uint32_t fast_bytes;
-#	define LZMA_FAST_BYTES_MIN                 5
-#	define LZMA_FAST_BYTES_MAX                 273
-#	define LZMA_FAST_BYTES_DEFAULT             128
+	uint32_t nice_len;
 
 	/** Match finder ID */
-	lzma_match_finder match_finder;
+	lzma_match_finder mf;
 
 	/**
-	 * \brief       Match finder cycles
+	 * \brief       Maximum search depth in the match finder
 	 *
-	 * Higher values give slightly better compression ratio but
-	 * decrease speed. Use special value 0 to let liblzma use
-	 * match-finder-dependent default value.
+	 * For every input byte, match finder searches through the hash chain
+	 * or binary tree in a loop, each iteration going one step deeper in
+	 * the chain or tree. The searching stops if
+	 *  - a match of at least nice_len bytes long is found;
+	 *  - all match candidates from the hash chain or binary tree have
+	 *    been checked; or
+	 *  - maximum search depth is reached.
 	 *
-	 * \todo        Write much better description.
+	 * Maximum search depth is needed to prevent the match finder from
+	 * wasting too much time in case there are lots of short match
+	 * candidates. On the other hand, stopping the search before all
+	 * candidates have been checked can reduce compression ratio.
+	 *
+	 * Setting depth to zero tells liblzma to use an automatic default
+	 * value, that depends on the selected match finder and nice_len.
+	 * The default is in the range [10, 200] or so (it may vary between
+	 * liblzma versions).
+	 *
+	 * Using a bigger depth value than the default can increase
+	 * compression ratio in some cases. There is no strict maximum value,
+	 * but high values (thousands or millions) should be used with care:
+	 * the encoder could remain fast enough with typical input, but
+	 * malicious input could cause the match finder to slow down
+	 * dramatically, possibly creating a denial of service attack.
 	 */
-	uint32_t match_finder_cycles;
+	uint32_t depth;
+
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. You should not touch these, because the names
+	 * of these variables may change. These are and will never be used
+	 * with the currently supported options, so it is safe to leave these
+	 * uninitialized.
+	 */
+	void *reserved_ptr1;
+	void *reserved_ptr2;
+	uint32_t reserved_int1;
+	uint32_t reserved_int2;
+	uint32_t reserved_int3;
+	uint32_t reserved_int4;
+	uint32_t reserved_int5;
+	uint32_t reserved_int6;
+	uint32_t reserved_int7;
+	uint32_t reserved_int8;
+	lzma_reserved_enum reserved_enum1;
+	lzma_reserved_enum reserved_enum2;
+	lzma_reserved_enum reserved_enum3;
+	lzma_reserved_enum reserved_enum4;
 
 } lzma_options_lzma;
 
 
 /**
- * \brief       Available LZMA encoding modes
+ * \brief       Set a compression preset to lzma_options_lzma structure
  *
- * Pointer to an array containing the list of available encoding modes.
- *
- * This variable is available only if LZMA encoder has been enabled.
- */
-extern const lzma_mode *const lzma_available_modes;
-
-
-/**
- * \brief       Available match finders
- *
- * Pointer to an array containing the list of available match finders.
- * The last element is LZMA_MF_INVALID.
- *
- * This variable is available only if LZMA encoder has been enabled.
- */
-extern const lzma_match_finder *const lzma_available_match_finders;
-
-
-/**
- * \brief       Table of presets for the LZMA filter
- *
- * lzma_presets[0] is the fastest and lzma_preset_lzma[8] is the slowest.
- * These presets match the switches -1 .. -9 of the lzma command line tool
+ * 0 is the fastest and 9 is the slowest. These match the switches -0 .. -9
+ * of the xz command line tool. In addition, it is possible to bitwise-or
+ * flags to the preset. Currently only LZMA_PRESET_EXTREME is supported.
+ * The flags are defined in container.h, because the flags are used also
+ * with lzma_easy_encoder().
  *
  * The preset values are subject to changes between liblzma versions.
  *
- * This variable is available only if LZMA encoder has been enabled.
+ * This function is available only if LZMA1 or LZMA2 encoder has been enabled
+ * when building liblzma.
  */
-extern const lzma_options_lzma lzma_preset_lzma[9];
+extern lzma_bool lzma_lzma_preset(lzma_options_lzma *options, uint32_t preset);

src/liblzma/api/lzma/memlimit.h

@@ -1,157 +0,0 @@
-/**
- * \file        lzma/memlimit.h
- * \brief       Memory usage limitter
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Opaque data type used with the memory usage limitting functions
- */
-typedef struct lzma_memlimit_s lzma_memlimit;
-
-
-/**
- * \brief       Allocates and initializes a new lzma_memlimit structure
- *
- * It is easy to make liblzma to use huge amounts of memory. This can
- * be a problem especially with the decoder, since it a file requiring
- * huge amounts of memory to uncompress could allow even a denial of
- * service attack if the memory usage wasn't limited.
- *
- * liblzma provides a set of functions to control memory usage. Pointers
- * to these functions can be used in lzma_allocator structure, which makes
- * it easy to limit memory usage with liblzma.
- *
- * The memory limitter functions are not tied to limitting memory usage
- * with liblzma itself. You can use them with anything you like.
- *
- * In multi-threaded applications, only one thread at once may use the same
- * lzma_memlimit structure. If there is a need, this limitation may
- * be removed in future versions without breaking the libary API/ABI.
- *
- * \param       limit   Initial memory usage limit in bytes
- *
- * \return      Pointer to allocated and initialized lzma_memlimit
- *              structure. On error, NULL is returned. The reason behind
- *              an error is either that malloc() failed or that the given
- *              limit was so small that it didn't allow allocating even
- *              the lzma_memlimit structure itself.
- *
- * \note        Excluding lzma_memlimit_usage(), the functions whose name begin
- *              lzma_memlimit_ can be used even if lzma_init() hasn't been
- *              called.
- */
-extern lzma_memlimit *lzma_memlimit_create(size_t limit);
-
-
-/**
- * \brief       Sets a new memory usage limit
- *
- * \param       mem     Pointer to a lzma_memlimit structure returned
- *                      earlier by lzma_memry_limit_create().
- * \param       limit   New memory usage limit
- *
- * The new usage limit may be smaller than the amount of memory currently
- * allocated via *mem: New allocations will fail until enough memory has
- * been freed or a new limit is set, but the existing allocatations will
- * stay untouched.
- */
-extern void lzma_memlimit_set(lzma_memlimit *mem, size_t limit);
-
-
-/**
- * \brief       Gets the current memory usage limit
- */
-extern size_t lzma_memlimit_get(const lzma_memlimit *mem);
-
-
-/**
- * \brief       Gets the amount of currently allocated memory
- *
- * \note        This value includes the sizes of some helper structures,
- *              thus it will always be larger than the total number of
- *              bytes allocated via lzma_memlimit_alloc().
- */
-extern size_t lzma_memlimit_used(const lzma_memlimit *mem);
-
-
-/**
- * \brief       Allocates memory with malloc() if memory limit allows
- *
- * \param       mem     Pointer to a lzma_memlimit structure returned
- *                      earlier by lzma_memry_limit_create().
- * \param       nmemb   Number of elements to allocate. While liblzma always
- *                      sets this to one, this function still takes the
- *                      value of nmemb into account to keep the function
- *                      usable with zlib and libbzip2.
- * \param       size    Size of an element.
- *
- * \return      Pointer to memory allocated with malloc(nmemb * size),
- *              except if nmemb * size == 0 which returns malloc(1).
- *              On error, NULL is returned.
- *
- * \note        This function assumes that nmemb * size is at maximum of
- *              SIZE_MAX. If it isn't, an overflow will occur resulting
- *              invalid amount of memory being allocated.
- */
-extern void *lzma_memlimit_alloc(
-		lzma_memlimit *mem, size_t nmemb, size_t size);
-
-
-/**
- * \brief       Removes the pointer from memory limitting list
- *
- * \param       mem     Pointer to a lzma_memlimit structure returned
- *                      earlier by lzma_memry_limit_create().
- * \param       ptr     Pointer returned earlier by lzma_memlimit_alloc().
- *
- * This function removes ptr from the internal list and decreases the
- * counter of used memory accordingly. The ptr itself isn't freed. This is
- * useful when Extra Records allocated by liblzma using lzma_memlimit
- * are needed by the application and must not be freed when the
- * lzma_memlimit structure is destroyed.
- *
- * It is OK to call this function with ptr that hasn't been allocated with
- * lzma_memlimit_alloc(). In that case, this has no effect other than wasting
- * a few CPU cycles.
- */
-extern void lzma_memlimit_detach(lzma_memlimit *mem, void *ptr);
-
-
-/**
- * \brief       Frees memory and updates the memory limit list
- *
- * This is like lzma_memlimit_detach() but also frees the given pointer.
- */
-extern void lzma_memlimit_free(lzma_memlimit *mem, void *ptr);
-
-
-/**
- * \brief       Frees the memory allocated for and by the memory usage limitter
- *
- * \param       mem             Pointer to memory limitter
- * \param       free_allocated  If this is non-zero, all the memory allocated
- *                              by lzma_memlimit_alloc() using *mem is also
- *                              freed if it hasn't already been freed with
- *                              lzma_memlimit_free(). Usually this should be
- *                              set to true.
- */
-extern void lzma_memlimit_end(
-		lzma_memlimit *mem, lzma_bool free_allocated);

src/liblzma/api/lzma/metadata.h

@@ -1,100 +0,0 @@
-/**
- * \file        lzma/metadata.h
- * \brief       Metadata handling
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Information stored into a Metadata Block
- *
- * This structure holds all the information that can be stored to
- * a Metadata Block.
- */
-typedef struct {
-	/**
-	 * \brief       Size of Header Metadata Block
-	 */
-	lzma_vli header_metadata_size;
-
-	/**
-	 * \brief       Total Size of the Stream
-	 */
-	lzma_vli total_size;
-
-	/**
-	 * \brief       Uncompressed Size of the Stream
-	 */
-	lzma_vli uncompressed_size;
-
-	/**
-	 * \brief       Index of the Blocks stored in the Stream
-	 */
-	lzma_index *index;
-
-	/**
-	 * \brief       Extra information
-	 */
-	lzma_extra *extra;
-
-} lzma_metadata;
-
-
-/**
- * \brief       Calculate the encoded size of Metadata
- *
- * \return      Uncompressed size of the Metadata in encoded form. This value
- *              may be passed to Block encoder as Uncompressed Size when using
- *              Metadata filter. On error, zero is returned.
- */
-extern lzma_vli lzma_metadata_size(const lzma_metadata *metadata);
-
-
-/**
- * \brief       Initializes Metadata encoder
- *
- * \param       coder       Pointer to a pointer to hold Metadata encoder's
- *                          internal state. Original value is ignored, thus
- *                          you don't need to initialize the pointer.
- * \param       allocator   Custom memory allocator; usually NULL.
- * \param       metadata    Pointer to Metadata to encoded
- *
- * \return      - LZMA_OK: Initialization succeeded.
- *              - LZMA_MEM_ERROR: Cannot allocate memory for *coder.
- *
- * The initialization function makes internal copy of the *metadata structure.
- * However, the linked lists metadata->index and metadata->extra are NOT
- * copied. Thus, the application may destroy *metadata after initialization
- * if it likes, but not Index or Extra.
- */
-extern lzma_ret lzma_metadata_encoder(lzma_stream *strm,
-		lzma_options_block *options, const lzma_metadata *metadata);
-
-
-/**
- * \brief       Initializes Metadata decoder
- *
- * \param       want_extra    If this is true, Extra Records will be stored
- *                            to metadata->extra. If this is false, Extra
- *                            Records will be parsed but not stored anywhere,
- *                            metadata->extra will be set to NULL.
- */
-extern lzma_ret lzma_metadata_decoder(
-		lzma_stream *strm, lzma_options_block *options,
-		lzma_metadata *metadata, lzma_bool want_extra);

src/liblzma/api/lzma/raw.h

@@ -1,72 +0,0 @@
-/**
- * \file        lzma/raw.h
- * \brief       Raw encoder and decoder
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Initializes raw encoder
- *
- * This function may be useful when implementing custom file formats.
- *
- * \param       strm    Pointer to properly prepared lzma_stream
- * \param       options Array of lzma_options_filter structures.
- *                      The end of the array must be marked with
- *                      .id = LZMA_VLI_VALUE_UNKNOWN. The minimum
- *                      number of filters is zero; the maximum is
- *                      determined by available memory.
- * \param       uncompressed_size
- *                      Size of the uncompressed data. If it is unknown,
- *                      use LZMA_VLI_VALUE_UNKNOWN. You need to give the
- *                      same value to the raw decoder to decode the data.
- * \param       allow_implicit
- *                      If true, an implicit Copy or Subblock filter should be
- *                      automatically added when needed. If this is false and
- *                      an implicit filter would be needed, LZMA_PROG_ERROR is
- *                      returned.
- *
- * The `action' with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the
- * filter chain support it), or LZMA_FINISH.
- *
- * \return      - LZMA_OK
- *              - LZMA_MEM_ERROR
- *              - LZMA_HEADER_ERROR
- *              - LZMA_PROG_ERROR
- */
-extern lzma_ret lzma_raw_encoder(
-		lzma_stream *strm, const lzma_options_filter *options,
-		lzma_vli uncompressed_size, lzma_bool allow_implicit);
-
-
-/**
- * \brief       Initializes raw decoder
- *
- * The initialization of raw decoder goes similarly to raw encoder.
- *
- * The `action' with lzma_code() can be LZMA_RUN or LZMA_SYNC_FLUSH.
- *
- * \return      - LZMA_OK
- *              - LZMA_MEM_ERROR
- *              - LZMA_HEADER_ERROR
- *              - LZMA_PROG_ERROR
- */
-extern lzma_ret lzma_raw_decoder(
-		lzma_stream *strm, const lzma_options_filter *options,
-		lzma_vli uncompressed_size, lzma_bool allow_implicit);

src/liblzma/api/lzma/stream.h

@@ -1,178 +0,0 @@
-/**
- * \file        lzma/stream.h
- * \brief       .lzma Stream handling
- *
- * \author      Copyright (C) 1999-2006 Igor Pavlov
- * \author      Copyright (C) 2007 Lasse Collin
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation; either
- * version 2.1 of the License, or (at your option) any later version.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- */
-
-#ifndef LZMA_H_INTERNAL
-#	error Never include this file directly. Use <lzma.h> instead.
-#endif
-
-
-/**
- * \brief       Options for .lzma Stream encoder
- */
-typedef struct {
-	/**
-	 * \brief       Type of integrity Check
-	 *
-	 * The type of the integrity Check is stored into Stream Header
-	 * and Stream Footer. The same Check is used for all Blocks in
-	 * the Stream.
-	 */
-	lzma_check_type check;
-
-	/**
-	 * \brief       Precense of CRC32 of the Block Header
-	 *
-	 * Set this to true if CRC32 of every Block Header should be
-	 * calculated and stored in the Block Header. This is recommended.
-	 *
-	 * This setting is stored into Stream Header and Stream Footer.
-	 */
-	lzma_bool has_crc32;
-
-	/**
-	 * \brief       Uncompressed Size in bytes
-	 *
-	 * This is somewhat advanced feature. Most users want to set this to
-	 * LZMA_VLI_VALUE_UNKNOWN to indicate unknown Uncompressed Size.
-	 *
-	 * If the Uncompressed Size of the Stream being encoded is known,
-	 * it can be stored to the beginning of the Stream. The details
-	 * differ for Single-Block and Multi-Block Streams:
-	 *  - With Single-Block Streams, the Uncompressed Size is stored to
-	 *    the Block Header and End of Payload Marker is omitted.
-	 *  - With Multi-Block Streams, the Uncompressed Size is stored to
-	 *    the Header Metadata Block. The Uncompressed Size of the Blocks
-	 *    will be unknown, because liblzma cannot predict how the
-	 *    application is going to split the data in Blocks.
-	 */
-	lzma_vli uncompressed_size;
-
-	/**
-	 * \brief       Alignment of the beginning of the Stream
-	 *
-	 * Certain filters handle data in bigger chunks than single bytes.
-	 * This affects two things:
-	 *   - Performance: aligned memory access is usually faster.
-	 *   - Further compression ratio in custom file formats: if you
-	 *     encode multiple Blocks with some non-compression filter
-	 *     such as LZMA_FILTER_POWERPC, it is a good idea to keep
-	 *     the inter-Block alignment correct to maximize compression
-	 *     ratio when all these Blocks are finally compressed as a
-	 *     single step.
-	 *
-	 * Usually the Stream is stored into its own file, thus
-	 * the alignment is usually zero.
-	 */
-	uint32_t alignment;
-
-	/**
-	 * \brief       Array of filters used to encode Data Blocks
-	 *
-	 * There can be at maximum of seven filters. The end of the array is
-	 * marked with .id = LZMA_VLI_VALUE_UNKNOWN. (That's why the array
-	 * has eight members.) Minimum number of filters is zero; in that
-	 * case, an implicit Copy filter is used.
-	 */
-	lzma_options_filter filters[8];
-
-	/**
-	 * \brief       Array of filters used to encode Metadata Blocks
-	 *
-	 * This is like filters[] but for Metadata Blocks. If Metadata
-	 * Blocks are compressed, they usually are compressed with
-	 * settings that require only little memory to uncompress e.g.
-	 * LZMA with 64 KiB dictionary.
-	 *
-	 * \todo        Recommend a preset.
-	 *
-	 * When liblzma sees that the Metadata Block would be very small
-	 * even in uncompressed form, it is not compressed no matter
-	 * what filter have been set here. This is to avoid possibly
-	 * increasing the size of the Metadata Block with bad compression,
-	 * and to avoid useless overhead of filters in uncompression phase.
-	 */
-	lzma_options_filter metadata_filters[8];
-
-	/**
-	 * \brief       Extra information in the Header Metadata Block
-	 */
-	lzma_extra *header;
-
-	/**
-	 * \brief       Extra information in the Footer Metadata Block
-	 *
-	 * It is enough to set this pointer any time before calling
-	 * lzma_code() with LZMA_FINISH as the second argument.
-	 */
-	lzma_extra *footer;
-
-} lzma_options_stream;
-
-
-/**
- * \brief       Initializes Single-Block .lzma Stream encoder
- *
- * This is the function that most developers are looking for. :-) It
- * compresses using the specified options without storing any extra
- * information.
- *
- * \todo        Describe that is_metadata is ignored, maybe something else.
- */
-extern lzma_ret lzma_stream_encoder_single(
-		lzma_stream *strm, const lzma_options_stream *options);
-
-
-/**
- * \brief       Initializes Multi-Block .lzma Stream encoder
- *
- */
-extern lzma_ret lzma_stream_encoder_multi(
-		lzma_stream *strm, const lzma_options_stream *options);
-
-
-/**
- * \brief       Initializes decoder for .lzma Stream
- *
- * \param       strm        Pointer to propertily prepared lzma_stream
- * \param       header      Pointer to hold a pointer to Extra Records read
- *                          from the Header Metadata Block. Use NULL if
- *                          you don't care about Extra Records.
- * \param       footer      Same as header, but for Footer Metadata Block.
- *
- * \return      - LZMA_OK: Initialization was successful.
- *              - LZMA_MEM_ERROR: Cannot allocate memory.
- *
- * If header and/or footer are not NULL, *header and/or *footer will be
- * initially set to NULL.
- *
- * The application can detect that Header Metadata Block has been completely
- * parsed when the decoder procudes some output the first time. If *header
- * is still NULL, there was no Extra field in the Header Metadata Block (or
- * the whole Header Metadata Block wasn't present at all).
- *
- * The application can detect that Footer Metadata Block has been parsed
- * completely when lzma_code() returns LZMA_STREAM_END. If *footer is still
- * NULL, there was no Extra field in the Footer Metadata Block.
- *
- * \note        If you use lzma_memory_limitter, the Extra Records will be
- *              allocated with it, and thus remain in the lzma_memory_limitter
- *              even after they get exported to the application via *header
- *              and *footer pointers.
- */
-extern lzma_ret lzma_stream_decoder(lzma_stream *strm,
-		lzma_extra **header, lzma_extra **footer);

src/liblzma/api/lzma/stream_flags.h

@@ -1,6 +1,6 @@
 /**
  * \file        lzma/stream_flags.h
- * \brief       .lzma Stream Header and Stream tail encoder and decoder
+ * \brief       .xz Stream Header and Stream Footer encoder and decoder
  *
  * \author      Copyright (C) 1999-2006 Igor Pavlov
  * \author      Copyright (C) 2007 Lasse Collin
@@ -22,121 +22,210 @@
 
 
 /**
- * \brief       Size of Stream Header
+ * \brief       Size of Stream Header and Stream Footer
  *
- * Magic Bytes (6) + Stream Flags (1) + CRC32 (4)
+ * Stream Header and Stream Footer have the same size and they are not
+ * going to change even if a newer version of the .xz file format is
+ * developed in future.
  */
-#define LZMA_STREAM_HEADER_SIZE (6 + 1 + 4)
+#define LZMA_STREAM_HEADER_SIZE 12
 
 
 /**
- * \brief       Size of Stream tail
- *
- * Because Stream Footer already has a defined meaning in the file format
- * specification, we use Stream tail to denote these two fields:
- * Stream Flags (1) + Magic Bytes (2)
- */
-#define LZMA_STREAM_TAIL_SIZE (1 + 2)
-
-
-/**
- * Options for encoding and decoding Stream Header and Stream tail
+ * Options for encoding and decoding Stream Header and Stream Footer
  */
 typedef struct {
 	/**
-	 * Type of the Check calculated from uncompressed data
+	 * \brief       Stream Flags format version
+	 *
+	 * To prevent API and ABI breakages if new features are needed in
+	 * Stream Header or Stream Footer, a version number is used to
+	 * indicate which fields in this structure are in use. For now,
+	 * version must always be zero. With non-zero version, the
+	 * lzma_stream_header_encode() and lzma_stream_footer_encode()
+	 * will return LZMA_OPTIONS_ERROR.
+	 *
+	 * lzma_stream_header_decode() and lzma_stream_footer_decode()
+	 * will always set this to the lowest value that supports all the
+	 * features indicated by the Stream Flags field. The application
+	 * must check that the version number set by the decoding functions
+	 * is supported by the application. Otherwise it is possible that
+	 * the application will decode the Stream incorrectly.
 	 */
-	lzma_check_type check;
+	uint32_t version;
 
 	/**
-	 * True if Block Headers have the CRC32 field. Note that the CRC32
-	 * field is always present in the Stream Header.
+	 * \brief       Backward Size
+	 *
+	 * Backward Size must be a multiple of four bytes. In this Stream
+	 * format version, Backward Size is the size of the Index field.
+	 *
+	 * Backward Size isn't actually part of the Stream Flags field, but
+	 * it is convenient to include in this structure anyway. Backward
+	 * Size is present only in the Stream Footer. There is no need to
+	 * initialize backward_size when encoding Stream Header.
+	 *
+	 * lzma_stream_header_decode() always sets backward_size to
+	 * LZMA_VLI_UNKNOWN so that it is convenient to use
+	 * lzma_stream_flags_compare() when both Stream Header and Stream
+	 * Footer have been decoded.
 	 */
-	lzma_bool has_crc32;
+	lzma_vli backward_size;
+#	define LZMA_BACKWARD_SIZE_MIN 4
+#	define LZMA_BACKWARD_SIZE_MAX (LZMA_VLI_C(1) << 34)
 
 	/**
-	 * True if the Stream is a Multi-Block Stream.
+	 * \brief       Check ID
+	 *
+	 * This indicates the type of the integrity check calculated from
+	 * uncompressed data.
 	 */
-	lzma_bool is_multi;
+	lzma_check check;
+
+	/*
+	 * Reserved space to allow possible future extensions without
+	 * breaking the ABI. You should not touch these, because the
+	 * names of these variables may change.
+	 *
+	 * (We will never be able to use all of these since Stream Flags
+	 * is just two bytes plus Backward Size of four bytes. But it's
+	 * nice to have the proper types when they are needed.)
+	 */
+	lzma_reserved_enum reserved_enum1;
+	lzma_reserved_enum reserved_enum2;
+	lzma_reserved_enum reserved_enum3;
+	lzma_reserved_enum reserved_enum4;
+	lzma_reserved_enum reserved_enum5;
+	lzma_reserved_enum reserved_enum6;
+	lzma_bool reserved_bool1;
+	lzma_bool reserved_bool2;
+	lzma_bool reserved_bool3;
+	lzma_bool reserved_bool4;
+	lzma_bool reserved_bool5;
+	lzma_bool reserved_bool6;
+	lzma_bool reserved_bool7;
+	lzma_bool reserved_bool8;
+	uint32_t reserved_int1;
+	uint32_t reserved_int2;
+	uint32_t reserved_int3;
+	uint32_t reserved_int4;
 
 } lzma_stream_flags;
 
 
-#define lzma_stream_flags_is_equal(a, b) \
-	((a).check == (b).check \
-		&& (a).has_crc32 == (b).has_crc32 \
-		&& (a).is_multi == (b).is_multi)
-
-
 /**
- * \brief       Encodes Stream Header
+ * \brief       Encode Stream Header
  *
- * Encoding of the Stream Header is done with a single call instead of
- * first initializing and then doing the actual work with lzma_code().
- *
- * \param       out         Beginning of the output buffer
- * \param       out_pos     out[*out_pos] is the next write position. This
- *                          is updated by the encoder.
- * \param       out_size    out[out_size] is the first byte to not write.
  * \param       options     Stream Header options to be encoded.
+ *                          options->backward_size is ignored and doesn't
+ *                          need to be initialized.
+ * \param       out         Beginning of the output buffer of
+ *                          LZMA_STREAM_HEADER_SIZE bytes.
  *
  * \return      - LZMA_OK: Encoding was successful.
+ *              - LZMA_OPTIONS_ERROR: options->version is not supported by
+ *                this liblzma version.
  *              - LZMA_PROG_ERROR: Invalid options.
- *              - LZMA_BUF_ERROR: Not enough output buffer space.
  */
 extern lzma_ret lzma_stream_header_encode(
-		uint8_t *out, const lzma_stream_flags *options);
+		const lzma_stream_flags *options, uint8_t *out)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Encodes Stream tail
+ * \brief       Encode Stream Footer
+ *
+ * \param       options     Stream Footer options to be encoded.
+ * \param       out         Beginning of the output buffer of
+ *                          LZMA_STREAM_HEADER_SIZE bytes.
+ *
+ * \return      - LZMA_OK: Encoding was successful.
+ *              - LZMA_OPTIONS_ERROR: options->version is not supported by
+ *                this liblzma version.
+ *              - LZMA_PROG_ERROR: Invalid options.
+ */
+extern lzma_ret lzma_stream_footer_encode(
+		const lzma_stream_flags *options, uint8_t *out)
+		lzma_attr_warn_unused_result;
+
+
+/**
+ * \brief       Decode Stream Header
  *
- * \param       footer      Pointer to a pointer that will hold the
- *                          allocated buffer. Caller must free it once
- *                          it isn't needed anymore.
- * \param       footer_size Pointer to a variable that will the final size
- *                          of the footer buffer.
- * \param       allocator   lzma_allocator for custom allocator functions.
- *                          Set to NULL to use malloc().
  * \param       options     Stream Header options to be encoded.
+ * \param       in          Beginning of the input buffer of
+ *                          LZMA_STREAM_HEADER_SIZE bytes.
  *
- * \return      - LZMA_OK: Success; *header and *header_size set.
- *              - LZMA_PROG_ERROR: *options is invalid.
- *              - LZMA_MEM_ERROR: Cannot allocate memory.
+ * options->backward_size is always set to LZMA_VLI_UNKNOWN. This is to
+ * help comparing Stream Flags from Stream Header and Stream Footer with
+ * lzma_stream_flags_compare().
+ *
+ * \return      - LZMA_OK: Decoding was successful.
+ *              - LZMA_FORMAT_ERROR: Magic bytes don't match, thus the given
+ *                buffer cannot be Stream Header.
+ *              - LZMA_DATA_ERROR: CRC32 doesn't match, thus the header
+ *                is corrupt.
+ *              - LZMA_OPTIONS_ERROR: Unsupported options are present
+ *                in the header.
+ *
+ * \note        When decoding .xz files that contain multiple Streams, it may
+ *              make sense to print "file format not recognized" only if
+ *              decoding of the Stream Header of the _first_ Stream gives
+ *              LZMA_FORMAT_ERROR. If non-first Stream Header gives
+ *              LZMA_FORMAT_ERROR, the message used for LZMA_DATA_ERROR is
+ *              probably more appropriate.
+ *
+ *              For example, Stream decoder in liblzma uses LZMA_DATA_ERROR if
+ *              LZMA_FORMAT_ERROR is returned by lzma_stream_header_decode()
+ *              when decoding non-first Stream.
  */
-extern lzma_ret lzma_stream_tail_encode(
-		uint8_t *out, const lzma_stream_flags *options);
+extern lzma_ret lzma_stream_header_decode(
+		lzma_stream_flags *options, const uint8_t *in)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Initializes Stream Header decoder
+ * \brief       Decode Stream Footer
  *
- * \param       strm        Pointer to lzma_stream used to pass input data
- * \param       options     Target structure for parsed results
+ * \param       options     Stream Header options to be encoded.
+ * \param       in          Beginning of the input buffer of
+ *                          LZMA_STREAM_HEADER_SIZE bytes.
  *
- * \return      - LZMA_OK: Successfully initialized
- *              - LZMA_MEM_ERROR: Cannot allocate memory
+ * \return      - LZMA_OK: Decoding was successful.
+ *              - LZMA_FORMAT_ERROR: Magic bytes don't match, thus the given
+ *                buffer cannot be Stream Footer.
+ *              - LZMA_DATA_ERROR: CRC32 doesn't match, thus the Stream Footer
+ *                is corrupt.
+ *              - LZMA_OPTIONS_ERROR: Unsupported options are present
+ *                in Stream Footer.
  *
- * The actual decoding is done with lzma_code() and freed with lzma_end().
+ * \note        If Stream Header was already decoded successfully, but
+ *              decoding Stream Footer returns LZMA_FORMAT_ERROR, the
+ *              application should probably report some other error message
+ *              than "file format not recognized", since the file more likely
+ *              is corrupt (possibly truncated). Stream decoder in liblzma
+ *              uses LZMA_DATA_ERROR in this situation.
  */
-extern lzma_ret lzma_stream_header_decoder(
-		lzma_stream *strm, lzma_stream_flags *options);
+extern lzma_ret lzma_stream_footer_decode(
+		lzma_stream_flags *options, const uint8_t *in)
+		lzma_attr_warn_unused_result;
 
 
 /**
- * \brief       Initializes Stream tail decoder
+ * \brief       Compare two lzma_stream_flags structures
  *
- * \param       strm        Pointer to lzma_stream used to pass input data
- * \param       options     Target structure for parsed results.
- * \param       decode_uncompressed_size
- *                          Set to true if the first field to decode is
- *                          Uncompressed Size. Set to false if the first
- *                          field to decode is Backward Size.
+ * backward_size values are compared only if both are not
+ * LZMA_VLI_UNKNOWN.
  *
- * \return      - LZMA_OK: Successfully initialized
- *              - LZMA_MEM_ERROR: Cannot allocate memory
- *
- * The actual decoding is done with lzma_code() and freed with lzma_end().
+ * \return      - LZMA_OK: Both are equal. If either had backward_size set
+ *                to LZMA_VLI_UNKNOWN, backward_size values were not
+ *                compared or validated.
+ *              - LZMA_DATA_ERROR: The structures differ.
+ *              - LZMA_OPTIONS_ERROR: version in either structure is greater
+ *                than the maximum supported version (currently zero).
+ *              - LZMA_PROG_ERROR: Invalid value, e.g. invalid check or
+ *                backward_size.
  */
-extern lzma_ret lzma_stream_tail_decoder(
-		lzma_stream *strm, lzma_stream_flags *options);
+extern lzma_ret lzma_stream_flags_compare(
+		const lzma_stream_flags *a, const lzma_stream_flags *b)
+		lzma_attr_pure;

src/liblzma/api/lzma/subblock.h

@@ -24,7 +24,7 @@
 /**
  * \brief       Filter ID
  *
- * Filter ID of the Subblock filter. This is used as lzma_options_filter.id.
+ * Filter ID of the Subblock filter. This is used as lzma_filter.id.
  */
 #define LZMA_FILTER_SUBBLOCK    LZMA_VLI_C(0x01)
 
@@ -95,9 +95,10 @@ typedef struct {
 	 * input_offset % alignment == output_offset % alignment
 	 *
 	 * The Subblock filter assumes that the first output byte will be
-	 * written to a position in the output stream that is properly aligned.
-	 *
-	 * FIXME desc
+	 * written to a position in the output stream that is properly
+	 * aligned. This requirement is automatically met when the start
+	 * offset of the Stream or Block is correctly told to Block or
+	 * Stream encoder.
 	 */
 	uint32_t alignment;
 #	define LZMA_SUBBLOCK_ALIGNMENT_MIN 1
@@ -161,16 +162,17 @@ typedef struct {
 	 *
 	 * When subfilter_mode is LZMA_SUBFILTER_NONE, the application may
 	 * put Subfilter options to subfilter_options structure, and then
-	 * set subfilter_mode to LZMA_SUBFILTER_SET. This implies setting
-	 * flush to true. No new input data will be read until the Subfilter
-	 * has been enabled. Once the Subfilter has been enabled, liblzma
-	 * will set subfilter_mode to LZMA_SUBFILTER_RUN.
+	 * set subfilter_mode to LZMA_SUBFILTER_SET. No new input data will
+	 * be read until the Subfilter has been enabled. Once the Subfilter
+	 * has been enabled, liblzma will set subfilter_mode to
+	 * LZMA_SUBFILTER_RUN.
 	 *
 	 * When subfilter_mode is LZMA_SUBFILTER_RUN, the application may
-	 * set subfilter_mode to LZMA_SUBFILTER_FINISH. No new input data
-	 * will be read until the Subfilter has been finished. Once the
-	 * Subfilter has been finished, liblzma will set subfilter_mode
-	 * to LZMA_SUBFILTER_NONE.
+	 * set subfilter_mode to LZMA_SUBFILTER_FINISH. All the input
+	 * currently available will be encoded before unsetting the
+	 * Subfilter. Application must not change the amount of available
+	 * input until the Subfilter has finished. Once the Subfilter has
+	 * finished, liblzma will set subfilter_mode to LZMA_SUBFILTER_NONE.
 	 *
 	 * If the intent is to have Subfilter enabled to the very end of
 	 * the data, it is not needed to separately disable Subfilter with
@@ -178,6 +180,11 @@ typedef struct {
 	 * of lzma_code() will make the Subblock encoder to disable the
 	 * Subfilter once all the data has been ran through the Subfilter.
 	 *
+	 * After the first call with LZMA_SYNC_FLUSH or LZMA_FINISH, the
+	 * application must not change subfilter_mode until LZMA_STREAM_END.
+	 * Setting LZMA_SUBFILTER_SET/LZMA_SUBFILTER_FINISH and
+	 * LZMA_SYNC_FLUSH/LZMA_FINISH _at the same time_ is fine.
+	 *
 	 * \note        This variable is ignored if allow_subfilters is false.
 	 */
 	lzma_subfilter_mode subfilter_mode;
@@ -192,6 +199,6 @@ typedef struct {
 	 *
 	 * \note        This variable is ignored if allow_subfilters is false.
 	 */
-	lzma_options_filter subfilter_options;
+	lzma_filter subfilter_options;
 
 } lzma_options_subblock;

src/liblzma/api/lzma/version.h

@@ -24,36 +24,36 @@
 /**
  * \brief       Compile-time version number
  *
- * The version number is of format xyyyuuus where
- *  - x is the major LZMA SDK version
- *  - yyy is the minor LZMA SDK version
- *  - uuu is LZMA Utils version (reset to zero every time SDK version
- *    is incremented)
+ * The version number is of format xyyyzzzs where
+ *  - x = major
+ *  - yyy = minor
+ *  - zzz = revision
  *  - s indicates stability: 0 = alpha, 1 = beta, 2 = stable
+ *
+ * The same xyyyzzz triplet is never reused with different stability levels.
+ * For example, if 5.1.0alpha has been released, there will never be 5.1.0beta
+ * or 5.1.0 stable.
+ *
+ * \note        The version number of liblzma has nothing to with
+ *              the version number of Igor Pavlov's LZMA SDK.
  */
-#define LZMA_VERSION UINT32_C(40420010)
+#define LZMA_VERSION UINT32_C(49990071)
 
 
 /**
  * \brief       liblzma version number as an integer
  *
- * This is the value of LZMA_VERSION macro at the compile time of liblzma.
+ * Returns the value of LZMA_VERSION macro at the compile time of liblzma.
  * This allows the application to compare if it was built against the same,
  * older, or newer version of liblzma that is currently running.
  */
-extern const uint32_t lzma_version_number;
+extern uint32_t lzma_version_number(void) lzma_attr_const;
 
 
 /**
- * \brief       Returns versions number of liblzma as a string
+ * \brief       Version number of liblzma as a string
  *
  * This function may be useful if you want to display which version of
- * libilzma your application is currently using.
- *
- * \return      Returns a pointer to a statically allocated string constant,
- *              which contains the version number of liblzma. The format of
- *              the version string is usually (but not necessarily) x.y.z
- *              e.g. "4.42.1". Alpha and beta versions contain a suffix
- *              ("4.42.0alpha").
+ * liblzma your application is currently using.
  */
-extern const char *const lzma_version_string;
+extern const char *lzma_version_string(void) lzma_attr_const;

src/liblzma/api/lzma/vli.h

@@ -24,12 +24,12 @@
 /**
  * \brief       Maximum supported value of variable-length integer
  */
-#define LZMA_VLI_VALUE_MAX (UINT64_MAX / 2)
+#define LZMA_VLI_MAX (UINT64_MAX / 2)
 
 /**
  * \brief       VLI value to denote that the value is unknown
  */
-#define LZMA_VLI_VALUE_UNKNOWN UINT64_MAX
+#define LZMA_VLI_UNKNOWN UINT64_MAX
 
 /**
  * \brief       Maximum supported length of variable length integers
@@ -47,12 +47,11 @@
  * \brief       Variable-length integer type
  *
  * This will always be unsigned integer. Valid VLI values are in the range
- * [0, LZMA_VLI_VALUE_MAX]. Unknown value is indicated with
- * LZMA_VLI_VALUE_UNKNOWN, which is the maximum value of the underlaying
- * integer type (this feature is useful in several situations).
+ * [0, LZMA_VLI_MAX]. Unknown value is indicated with LZMA_VLI_UNKNOWN,
+ * which is the maximum value of the underlaying integer type.
  *
  * In future, even if lzma_vli is typdefined to something else than uint64_t,
- * it is guaranteed that 2 * LZMA_VLI_VALUE_MAX will not overflow lzma_vli.
+ * it is guaranteed that 2 * LZMA_VLI_MAX will not overflow lzma_vli.
  * This simplifies integer overflow detection.
  */
 typedef uint64_t lzma_vli;
@@ -68,115 +67,57 @@ typedef uint64_t lzma_vli;
  *              indicates unknown value.
  */
 #define lzma_vli_is_valid(vli) \
-	((vli) <= LZMA_VLI_VALUE_MAX || (vli) == LZMA_VLI_VALUE_UNKNOWN)
-
-
-/**
- * \brief       Sets VLI to given value with error checking
- *
- * \param       dest    Target variable which must have type of lzma_vli.
- * \param       src     New value to be stored to dest.
- * \param       limit   Maximum allowed value for src.
- *
- * \return      False on success, true on error. If an error occurred,
- *              dest is left in undefined state (i.e. it's possible that
- *              it will be different in newer liblzma versions).
- */
-#define lzma_vli_set_lim(dest, src, limit) \
-	((src) > (limit) || ((dest) = (src)) > (limit))
-
-/**
- * \brief
- */
-#define lzma_vli_add_lim(dest, src, limit) \
-	((src) > (limit) || ((dest) += (src)) > (limit))
-
-#define lzma_vli_add2_lim(dest, src1, src2, limit) \
-	(lzma_vli_add_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit))
-
-#define lzma_vli_add3_lim(dest, src1, src2, src3, limit) \
-	(lzma_vli_add_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit) \
-		|| lzma_vli_add_lim(dest, src3, limit))
-
-#define lzma_vli_add4_lim(dest, src1, src2, src3, src4, limit) \
-	(lzma_vli_add_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit) \
-		|| lzma_vli_add_lim(dest, src3, limit) \
-		|| lzma_vli_add_lim(dest, src4, limit))
-
-#define lzma_vli_sum_lim(dest, src1, src2, limit) \
-	(lzma_vli_set_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit))
-
-#define lzma_vli_sum3_lim(dest, src1, src2, src3, limit) \
-	(lzma_vli_set_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit) \
-		|| lzma_vli_add_lim(dest, src3, limit))
-
-#define lzma_vli_sum4_lim(dest, src1, src2, src3, src4, limit) \
-	(lzma_vli_set_lim(dest, src1, limit) \
-		|| lzma_vli_add_lim(dest, src2, limit) \
-		|| lzma_vli_add_lim(dest, src3, limit) \
-		|| lzma_vli_add_lim(dest, src4, limit))
-
-#define lzma_vli_set(dest, src) lzma_vli_set_lim(dest, src, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_add(dest, src) lzma_vli_add_lim(dest, src, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_add2(dest, src1, src2) \
-	lzma_vli_add2_lim(dest, src1, src2, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_add3(dest, src1, src2, src3) \
-	lzma_vli_add3_lim(dest, src1, src2, src3, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_add4(dest, src1, src2, src3, src4) \
-	lzma_vli_add4_lim(dest, src1, src2, src3, src4, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_sum(dest, src1, src2) \
-	lzma_vli_sum_lim(dest, src1, src2, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_sum3(dest, src1, src2, src3) \
-	lzma_vli_sum3_lim(dest, src1, src2, src3, LZMA_VLI_VALUE_MAX)
-
-#define lzma_vli_sum4(dest, src1, src2, src3, src4) \
-	lzma_vli_sum4_lim(dest, src1, src2, src3, src4, LZMA_VLI_VALUE_MAX)
+	((vli) <= LZMA_VLI_MAX || (vli) == LZMA_VLI_UNKNOWN)
 
 
 /**
  * \brief       Encodes variable-length integer
  *
- * In the new .lzma format, most integers are encoded in variable-length
- * representation. This saves space when smaller values are more likely
- * than bigger values.
+ * In the .xz format, most integers are encoded in a variable-length
+ * representation, which is sometimes called little endian base-128 encoding.
+ * This saves space when smaller values are more likely than bigger values.
  *
  * The encoding scheme encodes seven bits to every byte, using minimum
- * number of bytes required to represent the given value. In other words,
- * it puts 7-63 bits into 1-9 bytes. This implementation limits the number
- * of bits used to 63, thus num must be at maximum of INT64_MAX / 2. You
- * may use LZMA_VLI_VALUE_MAX for clarity.
+ * number of bytes required to represent the given value. Encodings that use
+ * non-minimum number of bytes are invalid, thus every integer has exactly
+ * one encoded representation. The maximum number of bits in a VLI is 63,
+ * thus the vli argument must be at maximum of UINT64_MAX / 2. You should
+ * use LZMA_VLI_MAX for clarity.
+ *
+ * This function has two modes: single-call and multi-call. Single-call mode
+ * encodes the whole integer at once; it is an error if the output buffer is
+ * too small. Multi-call mode saves the position in *vli_pos, and thus it is
+ * possible to continue encoding if the buffer becomes full before the whole
+ * integer has been encoded.
  *
  * \param       vli       Integer to be encoded
- * \param       vli_pos   How many bytes have already been written out. This
- *                        must be less than 9 before calling this function.
- * \param       vli_size  Minimum size that the variable-length representation
- *                        must take. This is useful if you want to use
- *                        variable-length integers as padding. Usually you want
- *                        to set this to zero. The maximum allowed value is 9.
+ * \param       vli_pos   How many VLI-encoded bytes have already been written
+ *                        out. When starting to encode a new integer, *vli_pos
+ *                        must be set to zero. To use single-call encoding,
+ *                        set vli_pos to NULL.
  * \param       out       Beginning of the output buffer
  * \param       out_pos   The next byte will be written to out[*out_pos].
  * \param       out_size  Size of the out buffer; the first byte into
  *                        which no data is written to is out[out_size].
  *
- * \return      - LZMA_OK: So far all OK, but the integer is not
+ * \return      Slightly different return values are used in multi-call and
+ *              single-call modes.
+ *
+ *              Single-call (vli_pos == NULL):
+ *              - LZMA_OK: Integer successfully encoded.
+ *              - LZMA_PROG_ERROR: Arguments are not sane. This can be due
+ *                to too little output space; single-call mode doesn't use
+ *                LZMA_BUF_ERROR, since the application should have checked
+ *                the encoded size with lzma_vli_size().
+ *
+ *              Multi-call (vli_pos != NULL):
+ *              - LZMA_OK: So far all OK, but the integer is not
  *                completely written out yet.
  *              - LZMA_STREAM_END: Integer successfully encoded.
- *              - LZMA_BUF_ERROR: No output space (*out_pos == out_size)
+ *              - LZMA_BUF_ERROR: No output space was provided.
  *              - LZMA_PROG_ERROR: Arguments are not sane.
  */
-extern lzma_ret lzma_vli_encode(
-		lzma_vli vli, size_t *lzma_restrict vli_pos, size_t vli_size,
+extern lzma_ret lzma_vli_encode(lzma_vli vli, size_t *lzma_restrict vli_pos,
 		uint8_t *lzma_restrict out, size_t *lzma_restrict out_pos,
 		size_t out_size);
 
@@ -184,22 +125,36 @@ extern lzma_ret lzma_vli_encode(
 /**
  * \brief       Decodes variable-length integer
  *
+ * Like lzma_vli_encode(), this function has single-call and multi-call modes.
+ *
  * \param       vli       Pointer to decoded integer. The decoder will
  *                        initialize it to zero when *vli_pos == 0, so
  *                        application isn't required to initialize *vli.
  * \param       vli_pos   How many bytes have already been decoded. When
  *                        starting to decode a new integer, *vli_pos must
- *                        be initialized to zero.
+ *                        be initialized to zero. To use single-call decoding,
+ *                        set this to NULL.
  * \param       in        Beginning of the input buffer
  * \param       in_pos    The next byte will be read from in[*in_pos].
  * \param       in_size   Size of the input buffer; the first byte that
  *                        won't be read is in[in_size].
  *
- * \return      - LZMA_OK: So far all OK, but the integer is not
+ * \return      Slightly different return values are used in multi-call and
+ *              single-call modes.
+ *
+ *              Single-call (vli_pos == NULL):
+ *              - LZMA_OK: Integer successfully decoded.
+ *              - LZMA_DATA_ERROR: Integer is corrupt. This includes hitting
+ *                the end of the input buffer before the whole integer was
+ *                decoded; providing no input at all will use LZMA_DATA_ERROR.
+ *              - LZMA_PROG_ERROR: Arguments are not sane.
+ *
+ *              Multi-call (vli_pos != NULL):
+ *              - LZMA_OK: So far all OK, but the integer is not
  *                completely decoded yet.
  *              - LZMA_STREAM_END: Integer successfully decoded.
- *              - LZMA_BUF_ERROR: No input data (*in_pos == in_size)
- *              - LZMA_DATA_ERROR: Integer is longer than nine bytes.
+ *              - LZMA_DATA_ERROR: Integer is corrupt.
+ *              - LZMA_BUF_ERROR: No input was provided.
  *              - LZMA_PROG_ERROR: Arguments are not sane.
  */
 extern lzma_ret lzma_vli_decode(lzma_vli *lzma_restrict vli,
@@ -208,37 +163,9 @@ extern lzma_ret lzma_vli_decode(lzma_vli *lzma_restrict vli,
 
 
 /**
- * \brief       Decodes variable-length integer reading buffer backwards
- *
- * The variable-length integer encoding is designed so that it can be read
- * either from the beginning to the end, or from the end to the beginning.
- * This feature is needed to make the Stream parseable backwards;
- * specifically, to read the Backward Size field in Stream Footer.
- *
- * \param       vli       Pointer to variable to hold the decoded integer.
- * \param       in        Beginning of the input buffer
- * \param       in_size   Number of bytes available in the in[] buffer.
- *                        On successful decoding, this is updated to match
- *                        the number of bytes used. (in[*in_size - 1] is the
- *                        first byte to process. After successful decoding,
- *                        in[*in_size] will point to the first byte of the
- *                        variable-length integer.)
- *
- * \return      - LZMA_OK: Decoding successful
- *              - LZMA_DATA_ERROR: No valid variable-length integer was found.
- *              - LZMA_BUF_ERROR: Not enough input. Note that in practice,
- *                this tends to be a sign of broken input, because the
- *                applications usually do give as much input to this function
- *                as the applications have available.
- */
-extern lzma_ret lzma_vli_reverse_decode(
-		lzma_vli *vli, const uint8_t *in, size_t *in_size);
-
-
-/**
- * \brief       Gets the minimum number of bytes required to encode vli
+ * \brief       Get the number of bytes required to encode a VLI
  *
  * \return      Number of bytes on success (1-9). If vli isn't valid,
  *              zero is returned.
  */
-extern size_t lzma_vli_size(lzma_vli vli);
+extern uint32_t lzma_vli_size(lzma_vli vli) lzma_attr_pure;

src/liblzma/check/Makefile.am

@@ -13,46 +13,36 @@ noinst_LTLIBRARIES = libcheck.la
 libcheck_la_SOURCES = \
 	check.c \
 	check.h \
-	check_init.c \
-	check_byteswap.h \
 	crc_macros.h
 libcheck_la_CPPFLAGS = \
 	-I@top_srcdir@/src/liblzma/api \
 	-I@top_srcdir@/src/liblzma/common
 
 if COND_CHECK_CRC32
-
-if COND_ASM_X86
-libcheck_la_SOURCES += crc32_x86.s
-else
-libcheck_la_SOURCES += crc32.c
-endif
-
 if COND_SMALL
-libcheck_la_SOURCES += crc32_init.c
+libcheck_la_SOURCES += crc32_small.c
 else
 libcheck_la_SOURCES += crc32_table.c crc32_table_le.h crc32_table_be.h
+if COND_ASM_X86
+libcheck_la_SOURCES += crc32_x86.S
+else
+libcheck_la_SOURCES += crc32_fast.c
+endif
 endif
-
 endif
-
 
 if COND_CHECK_CRC64
-
-if COND_ASM_X86
-libcheck_la_SOURCES += crc64_x86.s
-else
-libcheck_la_SOURCES += crc64.c
-endif
-
 if COND_SMALL
-libcheck_la_SOURCES += crc64_init.c
+libcheck_la_SOURCES += crc64_small.c
 else
 libcheck_la_SOURCES += crc64_table.c crc64_table_le.h crc64_table_be.h
+if COND_ASM_X86
+libcheck_la_SOURCES += crc64_x86.S
+else
+libcheck_la_SOURCES += crc64_fast.c
+endif
 endif
-
 endif
-
 
 if COND_CHECK_SHA256
 libcheck_la_SOURCES += sha256.c

src/liblzma/check/check.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       check.c
-/// \brief      Check sizes
+/// \brief      Single API to access different integrity checks
 //
 //  This code has been put into the public domain.
 //
@@ -13,99 +13,127 @@
 
 #include "check.h"
 
-// See the .lzma header format specification section 2.2.2.
-LZMA_API const uint32_t lzma_check_sizes[8] = { 0, 4, 4, 8, 16, 32, 32, 64 };
 
+extern LZMA_API lzma_bool
+lzma_check_is_supported(lzma_check type)
+{
+	if ((unsigned int)(type) > LZMA_CHECK_ID_MAX)
+		return false;
 
-LZMA_API const lzma_bool lzma_available_checks[LZMA_CHECK_ID_MAX + 1] = {
-	true,   // LZMA_CHECK_NONE
+	static const lzma_bool available_checks[LZMA_CHECK_ID_MAX + 1] = {
+		true,   // LZMA_CHECK_NONE
 
 #ifdef HAVE_CHECK_CRC32
-	true,
+		true,
 #else
-	false,
+		false,
 #endif
 
-	false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
 
 #ifdef HAVE_CHECK_CRC64
-	true,
+		true,
 #else
-	false,
+		false,
 #endif
 
-	false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
 
 #ifdef HAVE_CHECK_SHA256
-	true,
+		true,
 #else
-	false,
+		false,
 #endif
 
-	false,  // Reserved
-	false,  // Reserved
-};
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+		false,  // Reserved
+	};
+
+	return available_checks[(unsigned int)(type)];
+}
 
 
-extern lzma_ret
-lzma_check_init(lzma_check *check, lzma_check_type type)
+extern LZMA_API uint32_t
+lzma_check_size(lzma_check type)
 {
-	lzma_ret ret = LZMA_OK;
+	if ((unsigned int)(type) > LZMA_CHECK_ID_MAX)
+		return UINT32_MAX;
 
+	// See file-format.txt section 2.1.1.2.
+	static const uint8_t check_sizes[LZMA_CHECK_ID_MAX + 1] = {
+		0,
+		4, 4, 4,
+		8, 8, 8,
+		16, 16, 16,
+		32, 32, 32,
+		64, 64, 64
+	};
+
+	return check_sizes[(unsigned int)(type)];
+}
+
+
+extern void
+lzma_check_init(lzma_check_state *check, lzma_check type)
+{
 	switch (type) {
 	case LZMA_CHECK_NONE:
 		break;
 
 #ifdef HAVE_CHECK_CRC32
 	case LZMA_CHECK_CRC32:
-		check->crc32 = 0;
+		check->state.crc32 = 0;
 		break;
 #endif
 
 #ifdef HAVE_CHECK_CRC64
 	case LZMA_CHECK_CRC64:
-		check->crc64 = 0;
+		check->state.crc64 = 0;
 		break;
 #endif
 
 #ifdef HAVE_CHECK_SHA256
 	case LZMA_CHECK_SHA256:
-		lzma_sha256_init(&check->sha256);
+		lzma_sha256_init(check);
 		break;
 #endif
 
 	default:
-		if (type <= LZMA_CHECK_ID_MAX)
-			ret = LZMA_UNSUPPORTED_CHECK;
-		else
-			ret = LZMA_PROG_ERROR;
 		break;
 	}
 
-	return ret;
+	return;
 }
 
 
 extern void
-lzma_check_update(lzma_check *check, lzma_check_type type,
+lzma_check_update(lzma_check_state *check, lzma_check type,
 		const uint8_t *buf, size_t size)
 {
 	switch (type) {
 #ifdef HAVE_CHECK_CRC32
 	case LZMA_CHECK_CRC32:
-		check->crc32 = lzma_crc32(buf, size, check->crc32);
+		check->state.crc32 = lzma_crc32(buf, size, check->state.crc32);
 		break;
 #endif
 
 #ifdef HAVE_CHECK_CRC64
 	case LZMA_CHECK_CRC64:
-		check->crc64 = lzma_crc64(buf, size, check->crc64);
+		check->state.crc64 = lzma_crc64(buf, size, check->state.crc64);
 		break;
 #endif
 
 #ifdef HAVE_CHECK_SHA256
 	case LZMA_CHECK_SHA256:
-		lzma_sha256_update(buf, size, &check->sha256);
+		lzma_sha256_update(buf, size, check);
 		break;
 #endif
 
@@ -118,43 +146,30 @@ lzma_check_update(lzma_check *check, lzma_check_type type,
 
 
 extern void
-lzma_check_finish(lzma_check *check, lzma_check_type type)
+lzma_check_finish(lzma_check_state *check, lzma_check type)
 {
-#ifdef HAVE_CHECK_SHA256
-	if (type == LZMA_CHECK_SHA256)
-		lzma_sha256_finish(&check->sha256);
+	switch (type) {
+#ifdef HAVE_CHECK_CRC32
+	case LZMA_CHECK_CRC32:
+		check->buffer.u32[0] = integer_le_32(check->state.crc32);
+		break;
 #endif
 
-	return;
-}
-
-
-/*
-extern bool
-lzma_check_compare(
-		lzma_check *check1, lzma_check *check2, lzma_check_type type)
-{
-	bool ret;
-
-	switch (type) {
-	case LZMA_CHECK_NONE:
-		break;
-
-	case LZMA_CHECK_CRC32:
-		ret = check1->crc32 != check2->crc32;
-		break;
-
+#ifdef HAVE_CHECK_CRC64
 	case LZMA_CHECK_CRC64:
-		ret = check1->crc64 != check2->crc64;
+		check->buffer.u64[0] = integer_le_64(check->state.crc64);
 		break;
+#endif
+
+#ifdef HAVE_CHECK_SHA256
+	case LZMA_CHECK_SHA256:
+		lzma_sha256_finish(check);
+		break;
+#endif
 
 	default:
-		// Unsupported check
-		assert(type <= 7);
-		ret = false;
 		break;
 	}
 
-	return ret;
+	return;
 }
-*/

src/liblzma/check/check.h

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       check.h
-/// \brief      Prototypes for different check functions
+/// \brief      Internal API to different integrity check functions
 //
 //  This code has been put into the public domain.
 //
@@ -17,86 +17,79 @@
 #include "common.h"
 
 
+// Index hashing needs the best possible hash function (preferably
+// a cryptographic hash) for maximum reliability.
+#if defined(HAVE_CHECK_SHA256)
+#	define LZMA_CHECK_BEST LZMA_CHECK_SHA256
+#elif defined(HAVE_CHECK_CRC64)
+#	define LZMA_CHECK_BEST LZMA_CHECK_CRC64
+#else
+#	define LZMA_CHECK_BEST LZMA_CHECK_CRC32
+#endif
+
+
+/// \brief      Structure to hold internal state of the check being calculated
+///
+/// \note       This is not in the public API because this structure may
+///             change in future if new integrity check algorithms are added.
 typedef struct {
-	/// Internal state
-	uint32_t state[8];
+	/// Buffer to hold the final result and a temporary buffer for SHA256.
+	union {
+		uint8_t u8[64];
+		uint32_t u32[16];
+		uint64_t u64[8];
+	} buffer;
 
-	/// Temporary 8-byte aligned buffer to hold incomplete chunk.
-	/// After lzma_check_finish(), the first 32 bytes will contain
-	/// the final digest in big endian byte order.
-	uint8_t buffer[64];
+	/// Check-specific data
+	union {
+		uint32_t crc32;
+		uint64_t crc64;
 
-	/// Size of the message excluding padding
-	uint64_t size;
+		struct {
+			/// Internal state
+			uint32_t state[8];
 
-} lzma_sha256;
-
-
-/// \note       This is not in the public API because this structure will
-///             change in future.
-typedef union {
-	uint32_t crc32;
-	uint64_t crc64;
-	lzma_sha256 sha256;
-} lzma_check;
+			/// Size of the message excluding padding
+			uint64_t size;
+		} sha256;
+	} state;
+
+} lzma_check_state;
 
 
+/// lzma_crc32_table[0] is needed by LZ encoder so we need to keep
+/// the array two-dimensional.
 #ifdef HAVE_SMALL
-extern uint32_t lzma_crc32_table[8][256];
-extern uint64_t lzma_crc64_table[4][256];
+extern uint32_t lzma_crc32_table[1][256];
 #else
 extern const uint32_t lzma_crc32_table[8][256];
 extern const uint64_t lzma_crc64_table[4][256];
 #endif
 
-// Generic
 
-/// \brief      Initializes *check depending on type
+/// \brief      Initialize *check depending on type
 ///
 /// \return     LZMA_OK on success. LZMA_UNSUPPORTED_CHECK if the type is not
 ///             supported by the current version or build of liblzma.
 ///             LZMA_PROG_ERROR if type > LZMA_CHECK_ID_MAX.
-///
-extern lzma_ret lzma_check_init(lzma_check *check, lzma_check_type type);
+extern void lzma_check_init(lzma_check_state *check, lzma_check type);
 
-/// \brief      Updates *check
-///
-extern void lzma_check_update(lzma_check *check, lzma_check_type type,
+/// Update the check state
+extern void lzma_check_update(lzma_check_state *check, lzma_check type,
 		const uint8_t *buf, size_t size);
 
-/// \brief      Finishes *check
-///
-extern void lzma_check_finish(lzma_check *check, lzma_check_type type);
+/// Finish the check calculation and store the result to check->buffer.u8.
+extern void lzma_check_finish(lzma_check_state *check, lzma_check type);
 
 
-/*
-/// \brief      Compare two checks
-///
-/// \return     false if the checks are identical; true if they differ.
-///
-extern bool lzma_check_compare(
-		lzma_check *check1, lzma_check *check2, lzma_check_type type);
-*/
-
-
-// CRC32
-
-extern void lzma_crc32_init(void);
-
-
-// CRC64
-
-extern void lzma_crc64_init(void);
-
-
-// SHA256
-
-extern void lzma_sha256_init(lzma_sha256 *sha256);
+/// Prepare SHA-256 state for new input.
+extern void lzma_sha256_init(lzma_check_state *check);
 
+/// Update the SHA-256 hash state
 extern void lzma_sha256_update(
-		const uint8_t *buf, size_t size, lzma_sha256 *sha256);
-
-extern void lzma_sha256_finish(lzma_sha256 *sha256);
+		const uint8_t *buf, size_t size, lzma_check_state *check);
 
+/// Finish the SHA-256 calculation and store the result to check->buffer.u8.
+extern void lzma_sha256_finish(lzma_check_state *check);
 
 #endif

src/liblzma/check/check_init.c

@@ -1,37 +0,0 @@
-///////////////////////////////////////////////////////////////////////////////
-//
-/// \file       check_init.c
-/// \brief      Static initializations for integrity checks
-//
-//  This code has been put into the public domain.
-//
-//  This library is distributed in the hope that it will be useful,
-//  but WITHOUT ANY WARRANTY; without even the implied warranty of
-//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-//
-///////////////////////////////////////////////////////////////////////////////
-
-#include "check.h"
-
-
-extern LZMA_API void
-lzma_init_check(void)
-{
-#ifdef HAVE_SMALL
-	static bool already_initialized = false;
-	if (already_initialized)
-		return;
-
-#	ifdef HAVE_CHECK_CRC32
-	lzma_crc32_init();
-#	endif
-
-#	ifdef HAVE_CHECK_CRC64
-	lzma_crc64_init();
-#	endif
-
-	already_initialized = true;
-#endif
-
-	return;
-}

src/liblzma/check/crc32_init.c

@@ -1,58 +0,0 @@
-///////////////////////////////////////////////////////////////////////////////
-//
-/// \file       crc32_init.c
-/// \brief      CRC32 table initialization
-//
-//  This code is based on various public domain sources.
-//  This code has been put into the public domain.
-//
-//  This library is distributed in the hope that it will be useful,
-//  but WITHOUT ANY WARRANTY; without even the implied warranty of
-//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-//
-///////////////////////////////////////////////////////////////////////////////
-
-#ifdef HAVE_CONFIG_H
-#	include <config.h>
-#endif
-
-#include <sys/types.h>
-#include <inttypes.h>
-
-#ifdef WORDS_BIGENDIAN
-#	include "check_byteswap.h"
-#endif
-
-
-uint32_t lzma_crc32_table[8][256];
-
-
-extern void
-lzma_crc32_init(void)
-{
-	static const uint32_t poly32 = UINT32_C(0xEDB88320);
-
-	for (size_t s = 0; s < 8; ++s) {
-		for (size_t b = 0; b < 256; ++b) {
-			uint32_t r = s == 0 ? b : lzma_crc32_table[s - 1][b];
-
-			for (size_t i = 0; i < 8; ++i) {
-				if (r & 1)
-					r = (r >> 1) ^ poly32;
-				else
-					r >>= 1;
-			}
-
-			lzma_crc32_table[s][b] = r;
-		}
-	}
-
-#ifdef WORDS_BIGENDIAN
-	for (size_t s = 0; s < 8; ++s)
-		for (size_t b = 0; b < 256; ++b)
-			lzma_crc32_table[s][b]
-					= bswap_32(lzma_crc32_table[s][b]);
-#endif
-
-	return;
-}

src/liblzma/check/crc32_small.c

@@ -0,0 +1,54 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       crc32_small.c
+/// \brief      CRC32 calculation (size-optimized)
+//
+//  This code has been put into the public domain.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "check.h"
+
+
+uint32_t lzma_crc32_table[1][256];
+
+
+static void
+crc32_init(void)
+{
+	static const uint32_t poly32 = UINT32_C(0xEDB88320);
+
+	for (size_t b = 0; b < 256; ++b) {
+		uint32_t r = b;
+		for (size_t i = 0; i < 8; ++i) {
+			if (r & 1)
+				r = (r >> 1) ^ poly32;
+			else
+				r >>= 1;
+		}
+
+		lzma_crc32_table[0][b] = r;
+	}
+
+	return;
+}
+
+
+extern LZMA_API uint32_t
+lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc)
+{
+	mythread_once(crc32_init);
+
+	crc = ~crc;
+
+	while (size != 0) {
+		crc = lzma_crc32_table[0][*buf++ ^ (crc & 0xFF)] ^ (crc >> 8);
+		--size;
+	}
+
+	return ~crc;
+}

src/liblzma/check/crc32_table.c

@@ -11,9 +11,7 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#ifdef HAVE_CONFIG_H
-#	include <config.h>
-#endif
+#include "common.h"
 
 #ifdef WORDS_BIGENDIAN
 #	include "crc32_table_be.h"

src/liblzma/check/crc32_table_be.h

@@ -1,7 +1,5 @@
 /* This file has been automatically generated by crc32_tablegen.c. */
 
-#include <inttypes.h>
-
 const uint32_t lzma_crc32_table[8][256] = {
 	{
 		0x00000000, 0x96300777, 0x2C610EEE, 0xBA510999, 

src/liblzma/check/crc32_table_le.h

@@ -1,7 +1,5 @@
 /* This file has been automatically generated by crc32_tablegen.c. */
 
-#include <inttypes.h>
-
 const uint32_t lzma_crc32_table[8][256] = {
 	{
 		0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 

src/liblzma/check/crc32_tablegen.c

@@ -1,9 +1,9 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       crc32_tablegen.c
-/// \brief      Generates CRC32 crc32_table.c
+/// \brief      Generate crc32_table_le.h and crc32_table_be.h
 ///
-/// Compiling: gcc -std=c99 -o crc32_tablegen crc32_tablegen.c crc32_init.c
+/// Compiling: gcc -std=c99 -o crc32_tablegen crc32_tablegen.c
 /// Add -DWORDS_BIGENDIAN to generate big endian table.
 //
 //  This code has been put into the public domain.
@@ -14,24 +14,52 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#include <sys/types.h>
 #include <inttypes.h>
 #include <stdio.h>
 
-
-extern void lzma_crc32_init(void);
-
-extern uint32_t lzma_crc32_table[8][256];
+#ifdef WORDS_BIGENDIAN
+#	include "../../common/bswap.h"
+#endif
 
 
-int
-main()
+static uint32_t crc32_table[8][256];
+
+
+static void
+init_crc32_table(void)
 {
-	lzma_crc32_init();
+	static const uint32_t poly32 = UINT32_C(0xEDB88320);
 
+	for (size_t s = 0; s < 8; ++s) {
+		for (size_t b = 0; b < 256; ++b) {
+			uint32_t r = s == 0 ? b : crc32_table[s - 1][b];
+
+			for (size_t i = 0; i < 8; ++i) {
+				if (r & 1)
+					r = (r >> 1) ^ poly32;
+				else
+					r >>= 1;
+			}
+
+			crc32_table[s][b] = r;
+		}
+	}
+
+#ifdef WORDS_BIGENDIAN
+	for (size_t s = 0; s < 8; ++s)
+		for (size_t b = 0; b < 256; ++b)
+			crc32_table[s][b] = bswap_32(crc32_table[s][b]);
+#endif
+
+	return;
+}
+
+
+static void
+print_crc32_table(void)
+{
 	printf("/* This file has been automatically generated by "
 			"crc32_tablegen.c. */\n\n"
-			"#include <inttypes.h>\n\n"
 			"const uint32_t lzma_crc32_table[8][256] = {\n\t{");
 
 	for (size_t s = 0; s < 8; ++s) {
@@ -39,7 +67,7 @@ main()
 			if ((b % 4) == 0)
 				printf("\n\t\t");
 
-			printf("0x%08" PRIX32, lzma_crc32_table[s][b]);
+			printf("0x%08" PRIX32, crc32_table[s][b]);
 
 			if (b != 255)
 				printf(", ");
@@ -51,5 +79,14 @@ main()
 			printf("\n\t}, {");
 	}
 
+	return;
+}
+
+
+int
+main(void)
+{
+	init_crc32_table();
+	print_crc32_table();
 	return 0;
 }

src/liblzma/check/crc32_x86.S

@@ -45,7 +45,7 @@ init_table(void)
  */
 
 	.text
-	.global	lzma_crc32
+	.globl	lzma_crc32
 	.type	lzma_crc32, @function
 
 	.align	16
@@ -215,3 +215,12 @@ lzma_crc32:
 	ret
 
 	.size	lzma_crc32, .-lzma_crc32
+
+/*
+ * This is needed to support non-executable stack. It's ugly to
+ * use __linux__ here, but I don't know a way to detect when
+ * we are using GNU assembler.
+ */
+#if defined(__ELF__) && defined(__linux__)
+	.section	.note.GNU-stack,"",@progbits
+#endif

src/liblzma/check/crc64_init.c

@@ -13,14 +13,11 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #ifdef HAVE_CONFIG_H
-#	include <config.h>
+#	include "check.h"
 #endif
 
-#include <sys/types.h>
-#include <inttypes.h>
-
 #ifdef WORDS_BIGENDIAN
-#	include "check_byteswap.h"
+#	include "../../common/bswap.h"
 #endif
 
 

src/liblzma/check/crc64_small.c

@@ -0,0 +1,54 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       crc64_small.c
+/// \brief      CRC64 calculation (size-optimized)
+//
+//  This code has been put into the public domain.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "check.h"
+
+
+static uint64_t crc64_table[256];
+
+
+static void
+crc64_init(void)
+{
+	static const uint64_t poly64 = UINT64_C(0xC96C5795D7870F42);
+
+	for (size_t b = 0; b < 256; ++b) {
+		uint64_t r = b;
+		for (size_t i = 0; i < 8; ++i) {
+			if (r & 1)
+				r = (r >> 1) ^ poly64;
+			else
+				r >>= 1;
+		}
+
+		crc64_table[b] = r;
+	}
+
+	return;
+}
+
+
+extern LZMA_API uint64_t
+lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc)
+{
+	mythread_once(crc64_init);
+
+	crc = ~crc;
+
+	while (size != 0) {
+		crc = crc64_table[*buf++ ^ (crc & 0xFF)] ^ (crc >> 8);
+		--size;
+	}
+
+	return ~crc;
+}

src/liblzma/check/crc64_table.c

@@ -11,9 +11,7 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#ifdef HAVE_CONFIG_H
-#	include <config.h>
-#endif
+#include "common.h"
 
 #ifdef WORDS_BIGENDIAN
 #	include "crc64_table_be.h"

src/liblzma/check/crc64_table_be.h

@@ -1,7 +1,5 @@
 /* This file has been automatically generated by crc64_tablegen.c. */
 
-#include <inttypes.h>
-
 const uint64_t lzma_crc64_table[4][256] = {
 	{
 		UINT64_C(0x0000000000000000), UINT64_C(0x6F5FA703BE4C2EB3), 

src/liblzma/check/crc64_table_le.h

@@ -1,7 +1,5 @@
 /* This file has been automatically generated by crc64_tablegen.c. */
 
-#include <inttypes.h>
-
 const uint64_t lzma_crc64_table[4][256] = {
 	{
 		UINT64_C(0x0000000000000000), UINT64_C(0xB32E4CBE03A75F6F), 

src/liblzma/check/crc64_tablegen.c

@@ -1,9 +1,9 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       crc64_tablegen.c
-/// \brief      Generates CRC64 crc64_table.c
+/// \brief      Generate crc64_table_le.h and crc64_table_be.h
 ///
-/// Compiling: gcc -std=c99 -o crc64_tablegen crc64_tablegen.c crc64_init.c
+/// Compiling: gcc -std=c99 -o crc64_tablegen crc64_tablegen.c
 /// Add -DWORDS_BIGENDIAN to generate big endian table.
 //
 //  This code has been put into the public domain.
@@ -14,24 +14,52 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#include <sys/types.h>
 #include <inttypes.h>
 #include <stdio.h>
 
-
-extern void lzma_crc64_init(void);
-
-extern uint64_t lzma_crc64_table[4][256];
+#ifdef WORDS_BIGENDIAN
+#	include "../../common/bswap.h"
+#endif
 
 
-int
-main()
+static uint64_t crc64_table[4][256];
+
+
+extern void
+init_crc64_table(void)
 {
-	lzma_crc64_init();
+	static const uint64_t poly64 = UINT64_C(0xC96C5795D7870F42);
 
+	for (size_t s = 0; s < 4; ++s) {
+		for (size_t b = 0; b < 256; ++b) {
+			uint64_t r = s == 0 ? b : crc64_table[s - 1][b];
+
+			for (size_t i = 0; i < 8; ++i) {
+				if (r & 1)
+					r = (r >> 1) ^ poly64;
+				else
+					r >>= 1;
+			}
+
+			crc64_table[s][b] = r;
+		}
+	}
+
+#ifdef WORDS_BIGENDIAN
+	for (size_t s = 0; s < 4; ++s)
+		for (size_t b = 0; b < 256; ++b)
+			crc64_table[s][b] = bswap_64(crc64_table[s][b]);
+#endif
+
+	return;
+}
+
+
+static void
+print_crc64_table(void)
+{
 	printf("/* This file has been automatically generated by "
 			"crc64_tablegen.c. */\n\n"
-			"#include <inttypes.h>\n\n"
 			"const uint64_t lzma_crc64_table[4][256] = {\n\t{");
 
 	for (size_t s = 0; s < 4; ++s) {
@@ -40,7 +68,7 @@ main()
 				printf("\n\t\t");
 
 			printf("UINT64_C(0x%016" PRIX64 ")",
-					lzma_crc64_table[s][b]);
+					crc64_table[s][b]);
 
 			if (b != 255)
 				printf(", ");
@@ -52,5 +80,14 @@ main()
 			printf("\n\t}, {");
 	}
 
+	return;
+}
+
+
+int
+main(void)
+{
+	init_crc64_table();
+	print_crc64_table();
 	return 0;
 }

src/liblzma/check/crc64_x86.S

@@ -38,7 +38,7 @@ init_table(void)
  */
 
 	.text
-	.global	lzma_crc64
+	.globl	lzma_crc64
 	.type	lzma_crc64, @function
 
 	.align	16
@@ -200,4 +200,13 @@ lzma_crc64:
 	popl	%ebx
 	ret
 
-	.size	lzma_crc32, .-lzma_crc32
+	.size	lzma_crc64, .-lzma_crc64
+
+/*
+ * This is needed to support non-executable stack. It's ugly to
+ * use __linux__ here, but I don't know a way to detect when
+ * we are using GNU assembler.
+ */
+#if defined(__ELF__) && defined(__linux__)
+	.section	.note.GNU-stack,"",@progbits
+#endif

src/liblzma/check/crc_macros.h

@@ -12,7 +12,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #ifdef WORDS_BIGENDIAN
-#	include "check_byteswap.h"
+#	include "../../common/bswap.h"
 
 #	define A(x) ((x) >> 24)
 #	define B(x) (((x) >> 16) & 0xFF)

src/liblzma/check/sha256.c

@@ -20,7 +20,7 @@
 #include "check.h"
 
 #ifndef WORDS_BIGENDIAN
-#	include "check_byteswap.h"
+#	include "../../common/bswap.h"
 #endif
 
 // At least on x86, GCC is able to optimize this to a rotate instruction.
@@ -104,18 +104,18 @@ transform(uint32_t state[static 8], const uint32_t data[static 16])
 
 
 static void
-process(lzma_sha256 *sha256)
+process(lzma_check_state *check)
 {
 #ifdef WORDS_BIGENDIAN
-	transform(sha256->state, (uint32_t *)(sha256->buffer));
+	transform(check->state.sha256.state, check->buffer.u32);
 
 #else
 	uint32_t data[16];
 
 	for (size_t i = 0; i < 16; ++i)
-		data[i] = bswap_32(*((uint32_t*)(sha256->buffer) + i));
+		data[i] = bswap_32(check->buffer.u32[i]);
 
-	transform(sha256->state, data);
+	transform(check->state.sha256.state, data);
 #endif
 
 	return;
@@ -123,41 +123,41 @@ process(lzma_sha256 *sha256)
 
 
 extern void
-lzma_sha256_init(lzma_sha256 *sha256)
+lzma_sha256_init(lzma_check_state *check)
 {
 	static const uint32_t s[8] = {
 		0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A,
 		0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19,
 	};
 
-	memcpy(sha256->state, s, sizeof(s));
-	sha256->size = 0;
+	memcpy(check->state.sha256.state, s, sizeof(s));
+	check->state.sha256.size = 0;
 
 	return;
 }
 
 
 extern void
-lzma_sha256_update(const uint8_t *buf, size_t size, lzma_sha256 *sha256)
+lzma_sha256_update(const uint8_t *buf, size_t size, lzma_check_state *check)
 {
 	// Copy the input data into a properly aligned temporary buffer.
 	// This way we can be called with arbitrarily sized buffers
 	// (no need to be multiple of 64 bytes), and the code works also
 	// on architectures that don't allow unaligned memory access.
 	while (size > 0) {
-		const size_t copy_start = sha256->size & 0x3F;
+		const size_t copy_start = check->state.sha256.size & 0x3F;
 		size_t copy_size = 64 - copy_start;
 		if (copy_size > size)
 			copy_size = size;
 
-		memcpy(sha256->buffer + copy_start, buf, copy_size);
+		memcpy(check->buffer.u8 + copy_start, buf, copy_size);
 
 		buf += copy_size;
 		size -= copy_size;
-		sha256->size += copy_size;
+		check->state.sha256.size += copy_size;
 
-		if ((sha256->size & 0x3F) == 0)
-			process(sha256);
+		if ((check->state.sha256.size & 0x3F) == 0)
+			process(check);
 	}
 
 	return;
@@ -165,38 +165,38 @@ lzma_sha256_update(const uint8_t *buf, size_t size, lzma_sha256 *sha256)
 
 
 extern void
-lzma_sha256_finish(lzma_sha256 *sha256)
+lzma_sha256_finish(lzma_check_state *check)
 {
 	// Add padding as described in RFC 3174 (it describes SHA-1 but
 	// the same padding style is used for SHA-256 too).
-	size_t pos = sha256->size & 0x3F;
-	sha256->buffer[pos++] = 0x80;
+	size_t pos = check->state.sha256.size & 0x3F;
+	check->buffer.u8[pos++] = 0x80;
 
 	while (pos != 64 - 8) {
 		if (pos == 64) {
-			process(sha256);
+			process(check);
 			pos = 0;
 		}
 
-		sha256->buffer[pos++] = 0x00;
+		check->buffer.u8[pos++] = 0x00;
 	}
 
 	// Convert the message size from bytes to bits.
-	sha256->size *= 8;
+	check->state.sha256.size *= 8;
 
 #ifdef WORDS_BIGENDIAN
-	*(uint64_t *)(sha256->buffer + 64 - 8) = sha256->size;
+	check->buffer.u64[(64 - 8) / 8] = check->state.sha256.size;
 #else
-	*(uint64_t *)(sha256->buffer + 64 - 8) = bswap_64(sha256->size);
+	check->buffer.u64[(64 - 8) / 8] = bswap_64(check->state.sha256.size);
 #endif
 
-	process(sha256);
+	process(check);
 
 	for (size_t i = 0; i < 8; ++i)
 #ifdef WORDS_BIGENDIAN
-		((uint32_t *)(sha256->buffer))[i] = sha256->state[i];
+		check->buffer.u32[i] = check->state.sha256.state[i];
 #else
-		((uint32_t *)(sha256->buffer))[i] = bswap_32(sha256->state[i]);
+		check->buffer.u32[i] = bswap_32(check->state.sha256.state[i]);
 #endif
 
 	return;

src/liblzma/common/Makefile.am

@@ -16,59 +16,40 @@ noinst_LTLIBRARIES = libcommon.la
 libcommon_la_CPPFLAGS = \
 	-I@top_srcdir@/src/liblzma/api \
 	-I@top_srcdir@/src/liblzma/check \
+	-I@top_srcdir@/src/liblzma/rangecoder \
 	-I@top_srcdir@/src/liblzma/lz \
 	-I@top_srcdir@/src/liblzma/lzma \
-	-I@top_srcdir@/src/liblzma/simple \
 	-I@top_srcdir@/src/liblzma/subblock \
-	-I@top_srcdir@/src/liblzma/rangecoder
+	-I@top_srcdir@/src/liblzma/delta \
+	-I@top_srcdir@/src/liblzma/simple
+
 libcommon_la_SOURCES = \
+	common.c \
 	common.h \
-	sysdefs.h \
-	allocator.c \
-	block_private.h \
-	extra.c \
-	features.c \
+	bsr.h \
+	block_util.c \
+	filter_common.c \
+	filter_common.h \
 	index.c \
-	info.c \
-	init.c \
-	memory_limitter.c \
-	memory_usage.c \
-	next_coder.c \
-	raw_common.c \
-	raw_common.h \
-	code.c \
-	version.c
-
-if COND_FILTER_COPY
-libcommon_la_SOURCES += \
-	copy_coder.c \
-	copy_coder.h
-endif
-
-if COND_FILTER_DELTA
-libcommon_la_SOURCES += \
-	delta_coder.c \
-	delta_coder.h
-endif
+	index.h \
+	stream_flags_common.c \
+	stream_flags_common.h \
+	vli_size.c
 
 if COND_MAIN_ENCODER
 libcommon_la_SOURCES += \
-	alignment.c \
-	auto_decoder.c \
 	alone_encoder.c \
 	block_encoder.c \
 	block_encoder.h \
 	block_header_encoder.c \
+	easy.c \
+	filter_encoder.c \
+	filter_encoder.h \
 	filter_flags_encoder.c \
-	init_encoder.c \
-	metadata_encoder.c \
-	metadata_encoder.h \
-	raw_encoder.c \
-	raw_encoder.h \
-	stream_common.c \
-	stream_common.h \
-	stream_encoder_single.c \
-	stream_encoder_multi.c \
+	index_encoder.c \
+	index_encoder.h \
+	stream_encoder.c \
+	stream_encoder.h \
 	stream_flags_encoder.c \
 	vli_encoder.c
 endif
@@ -77,18 +58,17 @@ if COND_MAIN_DECODER
 libcommon_la_SOURCES += \
 	alone_decoder.c \
 	alone_decoder.h \
+	auto_decoder.c \
 	block_decoder.c \
 	block_decoder.h \
 	block_header_decoder.c \
+	filter_decoder.c \
+	filter_decoder.h \
 	filter_flags_decoder.c \
-	init_decoder.c \
-	metadata_decoder.c \
-	metadata_decoder.h \
-	raw_decoder.c \
-	raw_decoder.h \
+	index_decoder.c \
+	index_hash.c \
 	stream_decoder.c \
+	stream_decoder.h \
 	stream_flags_decoder.c \
-	stream_flags_decoder.h \
-	vli_decoder.c \
-	vli_reverse_decoder.c
+	vli_decoder.c
 endif

src/liblzma/common/alignment.c

@@ -1,118 +0,0 @@
-///////////////////////////////////////////////////////////////////////////////
-//
-/// \file       alignment.c
-/// \brief      Calculates preferred alignments of different filters
-//
-//  Copyright (C) 2007 Lasse Collin
-//
-//  This library is free software; you can redistribute it and/or
-//  modify it under the terms of the GNU Lesser General Public
-//  License as published by the Free Software Foundation; either
-//  version 2.1 of the License, or (at your option) any later version.
-//
-//  This library is distributed in the hope that it will be useful,
-//  but WITHOUT ANY WARRANTY; without even the implied warranty of
-//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-//  Lesser General Public License for more details.
-//
-///////////////////////////////////////////////////////////////////////////////
-
-#include "common.h"
-
-
-extern LZMA_API uint32_t
-lzma_alignment_input(const lzma_options_filter *filters, uint32_t guess)
-{
-	for (size_t i = 0; filters[i].id != LZMA_VLI_VALUE_UNKNOWN; ++i) {
-		switch (filters[i].id) {
-		case LZMA_FILTER_COPY:
-		case LZMA_FILTER_DELTA:
-			// The same as the input, check the next filter.
-			continue;
-
-		case LZMA_FILTER_SUBBLOCK:
-			if (filters[i].options == NULL)
-				return LZMA_SUBBLOCK_ALIGNMENT_DEFAULT;
-			else
-				return ((const lzma_options_subblock *)(
-					filters[i].options))->alignment;
-
-		case LZMA_FILTER_X86:
-			return 1;
-
-		case LZMA_FILTER_ARMTHUMB:
-			return 2;
-
-		case LZMA_FILTER_POWERPC:
-		case LZMA_FILTER_ARM:
-		case LZMA_FILTER_SPARC:
-			return 4;
-
-		case LZMA_FILTER_IA64:
-			return 16;
-
-		case LZMA_FILTER_LZMA: {
-			const lzma_options_lzma *lzma = filters[i].options;
-			return 1 << MAX(lzma->pos_bits,
-					lzma->literal_pos_bits);
-		}
-
-		default:
-			return UINT32_MAX;
-		}
-	}
-
-	return guess;
-}
-
-
-extern LZMA_API uint32_t
-lzma_alignment_output(const lzma_options_filter *filters, uint32_t guess)
-{
-	// Check if there is only an implicit Copy filter.
-	if (filters[0].id == LZMA_VLI_VALUE_UNKNOWN)
-		return guess;
-
-	// Find the last filter in the chain.
-	size_t i = 0;
-	while (filters[i + 1].id != LZMA_VLI_VALUE_UNKNOWN)
-		++i;
-
-	do {
-		switch (filters[i].id) {
-		case LZMA_FILTER_COPY:
-		case LZMA_FILTER_DELTA:
-			// It's the same as the input alignment, so
-			// check the next filter.
-			continue;
-
-		case LZMA_FILTER_SUBBLOCK:
-			if (filters[i].options == NULL)
-				return LZMA_SUBBLOCK_ALIGNMENT_DEFAULT;
-			else
-				return ((const lzma_options_subblock *)(
-					filters[i].options))->alignment;
-
-		case LZMA_FILTER_X86:
-		case LZMA_FILTER_LZMA:
-			return 1;
-
-		case LZMA_FILTER_ARMTHUMB:
-			return 2;
-
-		case LZMA_FILTER_POWERPC:
-		case LZMA_FILTER_ARM:
-		case LZMA_FILTER_SPARC:
-			return 4;
-
-		case LZMA_FILTER_IA64:
-			return 16;
-
-		default:
-			return UINT32_MAX;
-		}
-	} while (i-- != 0);
-
-	// If we get here, we have the same alignment as the input data.
-	return guess;
-}

src/liblzma/common/allocator.c

@@ -1,57 +0,0 @@
-///////////////////////////////////////////////////////////////////////////////
-//
-/// \file       allocator.c
-/// \brief      Allocating and freeing memory
-//
-//  Copyright (C) 2007 Lasse Collin
-//
-//  This library is free software; you can redistribute it and/or
-//  modify it under the terms of the GNU Lesser General Public
-//  License as published by the Free Software Foundation; either
-//  version 2.1 of the License, or (at your option) any later version.
-//
-//  This library is distributed in the hope that it will be useful,
-//  but WITHOUT ANY WARRANTY; without even the implied warranty of
-//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-//  Lesser General Public License for more details.
-//
-///////////////////////////////////////////////////////////////////////////////
-
-#include "common.h"
-
-#undef lzma_free
-
-extern void * lzma_attribute((malloc))
-lzma_alloc(size_t size, lzma_allocator *allocator)
-{
-	// Some malloc() variants return NULL if called with size == 0.
-	if (size == 0)
-		size = 1;
-
-	void *ptr;
-
-	if (allocator != NULL && allocator->alloc != NULL)
-		ptr = allocator->alloc(allocator->opaque, 1, size);
-	else
-		ptr = malloc(size);
-
-#if !defined(NDEBUG) && defined(HAVE_MEMSET)
-	// This helps to catch some stupid mistakes.
-	if (ptr != NULL)
-		memset(ptr, 0xFD, size);
-#endif
-
-	return ptr;
-}
-
-
-extern void
-lzma_free(void *ptr, lzma_allocator *allocator)
-{
-	if (allocator != NULL && allocator->free != NULL)
-		allocator->free(allocator->opaque, ptr);
-	else
-		free(ptr);
-
-	return;
-}

src/liblzma/common/alone_decoder.c

@@ -19,6 +19,7 @@
 
 #include "alone_decoder.h"
 #include "lzma_decoder.h"
+#include "lz_decoder.h"
 
 
 struct lzma_coder_s {
@@ -32,9 +33,21 @@ struct lzma_coder_s {
 		SEQ_CODE,
 	} sequence;
 
+	/// Position in the header fields
 	size_t pos;
 
-	lzma_options_alone options;
+	/// Uncompressed size decoded from the header
+	lzma_vli uncompressed_size;
+
+	/// Memory usage limit
+	uint64_t memlimit;
+
+	/// Amount of memory actually needed (only an estimate)
+	uint64_t memusage;
+
+	/// Options decoded from the header needed to initialize
+	/// the LZMA decoder
+	lzma_options_lzma options;
 };
 
 
@@ -50,33 +63,36 @@ alone_decode(lzma_coder *coder,
 			&& (coder->sequence == SEQ_CODE || *in_pos < in_size))
 	switch (coder->sequence) {
 	case SEQ_PROPERTIES:
-		if (lzma_lzma_decode_properties(
-				&coder->options.lzma, in[*in_pos]))
-			return LZMA_DATA_ERROR;
+		if (lzma_lzma_lclppb_decode(&coder->options, in[*in_pos]))
+			return LZMA_FORMAT_ERROR;
 
 		coder->sequence = SEQ_DICTIONARY_SIZE;
 		++*in_pos;
 		break;
 
 	case SEQ_DICTIONARY_SIZE:
-		coder->options.lzma.dictionary_size
+		coder->options.dict_size
 				|= (size_t)(in[*in_pos]) << (coder->pos * 8);
 
 		if (++coder->pos == 4) {
-			// A hack to ditch tons of false positives: We allow
-			// only dictionary sizes that are a power of two.
-			// LZMA_Alone didn't create other kinds of files,
-			// although it's not impossible that files with
-			// other dictionary sizes exist. Well, if someone
-			// complains, this will be reconsidered.
-			size_t count = 0;
-			for (size_t i = 0; i < 32; ++i)
-				if (coder->options.lzma.dictionary_size
-						& (UINT32_C(1) << i))
-					++count;
+			if (coder->options.dict_size != UINT32_MAX) {
+				// A hack to ditch tons of false positives:
+				// We allow only dictionary sizes that are
+				// 2^n or 2^n + 2^(n-1). LZMA_Alone created
+				// only files with 2^n, but accepts any
+				// dictionary size. If someone complains, this
+				// will be reconsidered.
+				uint32_t d = coder->options.dict_size - 1;
+				d |= d >> 2;
+				d |= d >> 3;
+				d |= d >> 4;
+				d |= d >> 8;
+				d |= d >> 16;
+				++d;
 
-			if (count > 1)
-				return LZMA_DATA_ERROR;
+				if (d != coder->options.dict_size)
+					return LZMA_FORMAT_ERROR;
+			}
 
 			coder->pos = 0;
 			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
@@ -86,35 +102,39 @@ alone_decode(lzma_coder *coder,
 		break;
 
 	case SEQ_UNCOMPRESSED_SIZE:
-		coder->options.uncompressed_size
+		coder->uncompressed_size
 				|= (lzma_vli)(in[*in_pos]) << (coder->pos * 8);
-
-		if (++coder->pos == 8) {
-			// Another hack to ditch false positives: Assume that
-			// if the uncompressed size is known, it must be less
-			// than 256 GiB. Again, if someone complains, this
-			// will be reconsidered.
-			if (coder->options.uncompressed_size
-						!= LZMA_VLI_VALUE_UNKNOWN
-					&& coder->options.uncompressed_size
-						>= (LZMA_VLI_C(1) << 38))
-				return LZMA_DATA_ERROR;
-
-			coder->pos = 0;
-			coder->sequence = SEQ_CODER_INIT;
-		}
-
 		++*in_pos;
-		break;
+		if (++coder->pos < 8)
+			break;
+
+		// Another hack to ditch false positives: Assume that
+		// if the uncompressed size is known, it must be less
+		// than 256 GiB. Again, if someone complains, this
+		// will be reconsidered.
+		if (coder->uncompressed_size != LZMA_VLI_UNKNOWN
+				&& coder->uncompressed_size
+					>= (LZMA_VLI_C(1) << 38))
+			return LZMA_FORMAT_ERROR;
+
+		// Calculate the memory usage so that it is ready
+		// for SEQ_CODER_INIT.
+		coder->memusage = lzma_lzma_decoder_memusage(&coder->options)
+				+ LZMA_MEMUSAGE_BASE;
+
+		coder->pos = 0;
+		coder->sequence = SEQ_CODER_INIT;
+
+	// Fall through
 
 	case SEQ_CODER_INIT: {
-		// Two is enough because there won't be implicit filters.
+		if (coder->memusage > coder->memlimit)
+			return LZMA_MEMLIMIT_ERROR;
+
 		lzma_filter_info filters[2] = {
 			{
 				.init = &lzma_lzma_decoder_init,
-				.options = &coder->options.lzma,
-				.uncompressed_size = coder->options
-						.uncompressed_size,
+				.options = &coder->options,
 			}, {
 				.init = NULL,
 			}
@@ -125,10 +145,13 @@ alone_decode(lzma_coder *coder,
 		if (ret != LZMA_OK)
 			return ret;
 
-		coder->sequence = SEQ_CODE;
-	}
+		// Use a hack to set the uncompressed size.
+		lzma_lz_decoder_uncompressed(coder->next.coder,
+				coder->uncompressed_size);
 
-	// Fall through
+		coder->sequence = SEQ_CODE;
+		break;
+	}
 
 	case SEQ_CODE: {
 		return coder->next.code(coder->next.coder,
@@ -147,15 +170,36 @@ alone_decode(lzma_coder *coder,
 static void
 alone_decoder_end(lzma_coder *coder, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->next, allocator);
+	lzma_next_end(&coder->next, allocator);
 	lzma_free(coder, allocator);
 	return;
 }
 
 
 static lzma_ret
-alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator)
+alone_decoder_memconfig(lzma_coder *coder, uint64_t *memusage,
+		uint64_t *old_memlimit, uint64_t new_memlimit)
 {
+	if (new_memlimit != 0 && new_memlimit < coder->memusage)
+		return LZMA_MEMLIMIT_ERROR;
+
+	*memusage = coder->memusage;
+	*old_memlimit = coder->memlimit;
+	coder->memlimit = new_memlimit;
+
+	return LZMA_OK;
+}
+
+
+extern lzma_ret
+lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
+		uint64_t memlimit)
+{
+	lzma_next_coder_init(lzma_alone_decoder_init, next, allocator);
+
+	if (memlimit == 0)
+		return LZMA_PROG_ERROR;
+
 	if (next->coder == NULL) {
 		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
 		if (next->coder == NULL)
@@ -163,35 +207,28 @@ alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator)
 
 		next->code = &alone_decode;
 		next->end = &alone_decoder_end;
+		next->memconfig = &alone_decoder_memconfig;
 		next->coder->next = LZMA_NEXT_CODER_INIT;
 	}
 
 	next->coder->sequence = SEQ_PROPERTIES;
 	next->coder->pos = 0;
-	next->coder->options.lzma.dictionary_size = 0;
-	next->coder->options.uncompressed_size = 0;
+	next->coder->options.dict_size = 0;
+	next->coder->uncompressed_size = 0;
+	next->coder->memlimit = memlimit;
+	next->coder->memusage = LZMA_MEMUSAGE_BASE;
 
 	return LZMA_OK;
 }
 
 
-extern lzma_ret
-lzma_alone_decoder_init(lzma_next_coder *next, lzma_allocator *allocator)
-{
-	// We need to use _init2 because we don't pass any varadic args.
-	lzma_next_coder_init2(next, allocator, alone_decoder_init,
-			alone_decoder_init, allocator);
-}
-
-
 extern LZMA_API lzma_ret
-lzma_alone_decoder(lzma_stream *strm)
+lzma_alone_decoder(lzma_stream *strm, uint64_t memlimit)
 {
-	lzma_next_strm_init2(strm, alone_decoder_init,
-			alone_decoder_init, strm->allocator);
+	lzma_next_strm_init(lzma_alone_decoder_init, strm, memlimit);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
-	strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;
+	strm->internal->supported_actions[LZMA_FINISH] = true;
 
 	return LZMA_OK;
 }

src/liblzma/common/alone_decoder.h

@@ -17,8 +17,13 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
+#ifndef LZMA_ALONE_DECODER_H
+#define LZMA_ALONE_DECODER_H
+
 #include "common.h"
 
 
-extern lzma_ret lzma_alone_decoder_init(
-		lzma_next_coder *next, lzma_allocator *allocator);
+extern lzma_ret lzma_alone_decoder_init(lzma_next_coder *next,
+		lzma_allocator *allocator, uint64_t memlimit);
+
+#endif

src/liblzma/common/alone_encoder.c

@@ -21,19 +21,19 @@
 #include "lzma_encoder.h"
 
 
+#define ALONE_HEADER_SIZE (1 + 4 + 8)
+
+
 struct lzma_coder_s {
 	lzma_next_coder next;
 
 	enum {
-		SEQ_PROPERTIES,
-		SEQ_DICTIONARY_SIZE,
-		SEQ_UNCOMPRESSED_SIZE,
+		SEQ_HEADER,
 		SEQ_CODE,
 	} sequence;
 
-	size_t pos;
-
-	lzma_options_alone options;
+	size_t header_pos;
+	uint8_t header[ALONE_HEADER_SIZE];
 };
 
 
@@ -47,47 +47,23 @@ alone_encode(lzma_coder *coder,
 {
 	while (*out_pos < out_size)
 	switch (coder->sequence) {
-	case SEQ_PROPERTIES:
-		if (lzma_lzma_encode_properties(
-				&coder->options.lzma, out + *out_pos)) {
-			return LZMA_PROG_ERROR;
-		}
+	case SEQ_HEADER:
+		lzma_bufcpy(coder->header, &coder->header_pos,
+				ALONE_HEADER_SIZE,
+				out, out_pos, out_size);
+		if (coder->header_pos < ALONE_HEADER_SIZE)
+			return LZMA_OK;
 
-		coder->sequence = SEQ_DICTIONARY_SIZE;
-		++*out_pos;
+		coder->sequence = SEQ_CODE;
 		break;
 
-	case SEQ_DICTIONARY_SIZE:
-		out[*out_pos] = coder->options.lzma.dictionary_size
-				>> (coder->pos * 8);
-
-		if (++coder->pos == 4) {
-			coder->pos = 0;
-			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
-		}
-
-		++*out_pos;
-		break;
-
-	case SEQ_UNCOMPRESSED_SIZE:
-		out[*out_pos] = coder->options.uncompressed_size
-				>> (coder->pos * 8);
-
-		if (++coder->pos == 8) {
-			coder->pos = 0;
-			coder->sequence = SEQ_CODE;
-		}
-
-		++*out_pos;
-		break;
-
-	case SEQ_CODE: {
+	case SEQ_CODE:
 		return coder->next.code(coder->next.coder,
 				allocator, in, in_pos, in_size,
 				out, out_pos, out_size, action);
-	}
 
 	default:
+		assert(0);
 		return LZMA_PROG_ERROR;
 	}
 
@@ -98,7 +74,7 @@ alone_encode(lzma_coder *coder,
 static void
 alone_encoder_end(lzma_coder *coder, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->next, allocator);
+	lzma_next_end(&coder->next, allocator);
 	lzma_free(coder, allocator);
 	return;
 }
@@ -107,8 +83,10 @@ alone_encoder_end(lzma_coder *coder, lzma_allocator *allocator)
 // At least for now, this is not used by any internal function.
 static lzma_ret
 alone_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		const lzma_options_alone *options)
+		const lzma_options_lzma *options)
 {
+	lzma_next_coder_init(alone_encoder_init, next, allocator);
+
 	if (next->coder == NULL) {
 		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
 		if (next->coder == NULL)
@@ -119,23 +97,42 @@ alone_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 		next->coder->next = LZMA_NEXT_CODER_INIT;
 	}
 
-	// Initialize the LZMA_Alone coder variables.
-	next->coder->sequence = SEQ_PROPERTIES;
-	next->coder->pos = 0;
-	next->coder->options = *options;
+	// Basic initializations
+	next->coder->sequence = SEQ_HEADER;
+	next->coder->header_pos = 0;
 
-	// Verify uncompressed_size since the other functions assume that
-	// it is valid.
-	if (!lzma_vli_is_valid(next->coder->options.uncompressed_size))
-		return LZMA_PROG_ERROR;
+	// Encode the header:
+	// - Properties (1 byte)
+	if (lzma_lzma_lclppb_encode(options, next->coder->header))
+		return LZMA_OPTIONS_ERROR;
+
+	// - Dictionary size (4 bytes)
+	if (options->dict_size < LZMA_DICT_SIZE_MIN)
+		return LZMA_OPTIONS_ERROR;
+
+	// Round up to to the next 2^n or 2^n + 2^(n - 1) depending on which
+	// one is the next unless it is UINT32_MAX. While the header would
+	// allow any 32-bit integer, we do this to keep the decoder of liblzma
+	// accepting the resulting files.
+	uint32_t d = options->dict_size - 1;
+	d |= d >> 2;
+	d |= d >> 3;
+	d |= d >> 4;
+	d |= d >> 8;
+	d |= d >> 16;
+	if (d != UINT32_MAX)
+		++d;
+
+	integer_write_32(next->coder->header + 1, d);
+
+	// - Uncompressed size (always unknown and using EOPM)
+	memset(next->coder->header + 1 + 4, 0xFF, 8);
 
 	// Initialize the LZMA encoder.
 	const lzma_filter_info filters[2] = {
 		{
 			.init = &lzma_lzma_encoder_init,
-			.options = &next->coder->options.lzma,
-			.uncompressed_size = next->coder->options
-					.uncompressed_size,
+			.options = (void *)(options),
 		}, {
 			.init = NULL,
 		}
@@ -156,9 +153,9 @@ lzma_alone_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 
 
 extern LZMA_API lzma_ret
-lzma_alone_encoder(lzma_stream *strm, const lzma_options_alone *options)
+lzma_alone_encoder(lzma_stream *strm, const lzma_options_lzma *options)
 {
-	lzma_next_strm_init(strm, alone_encoder_init, options);
+	lzma_next_strm_init(alone_encoder_init, strm, options);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
 	strm->internal->supported_actions[LZMA_FINISH] = true;

src/liblzma/common/auto_decoder.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       auto_decoder.c
-/// \brief      Autodetect between .lzma Stream and LZMA_Alone formats
+/// \brief      Autodetect between .xz Stream and .lzma (LZMA_Alone) formats
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -17,16 +17,22 @@
 //
 ///////////////////////////////////////////////////////////////////////////////
 
-#include "common.h"
+#include "stream_decoder.h"
 #include "alone_decoder.h"
 
 
 struct lzma_coder_s {
+	/// Stream decoder or LZMA_Alone decoder
 	lzma_next_coder next;
 
-	lzma_extra **header;
-	lzma_extra **footer;
-	bool initialized;
+	uint64_t memlimit;
+	uint32_t flags;
+
+	enum {
+		SEQ_INIT,
+		SEQ_CODE,
+		SEQ_FINISH,
+	} sequence;
 };
 
 
@@ -36,42 +42,125 @@ auto_decode(lzma_coder *coder, lzma_allocator *allocator,
 		size_t in_size, uint8_t *restrict out,
 		size_t *restrict out_pos, size_t out_size, lzma_action action)
 {
-	if (!coder->initialized) {
+	switch (coder->sequence) {
+	case SEQ_INIT:
 		if (*in_pos >= in_size)
 			return LZMA_OK;
 
-		lzma_ret ret;
+		// Update the sequence now, because we want to continue from
+		// SEQ_CODE even if we return some LZMA_*_CHECK.
+		coder->sequence = SEQ_CODE;
 
-		if (in[*in_pos] == 0xFF)
-			ret = lzma_stream_decoder_init(&coder->next, allocator,
-					coder->header, coder->footer);
-		else
-			ret = lzma_alone_decoder_init(&coder->next, allocator);
+		// Detect the file format. For now this is simple, since if
+		// it doesn't start with 0xFD (the first magic byte of the
+		// new format), it has to be LZMA_Alone, or something that
+		// we don't support at all.
+		if (in[*in_pos] == 0xFD) {
+			return_if_error(lzma_stream_decoder_init(
+					&coder->next, allocator,
+					coder->memlimit, coder->flags));
+		} else {
+			return_if_error(lzma_alone_decoder_init(&coder->next,
+					allocator, coder->memlimit));
 
-		if (ret != LZMA_OK)
+			// If the application wants to know about missing
+			// integrity check or about the check in general, we
+			// need to handle it here, because LZMA_Alone decoder
+			// doesn't accept any flags.
+			if (coder->flags & LZMA_TELL_NO_CHECK)
+				return LZMA_NO_CHECK;
+
+			if (coder->flags & LZMA_TELL_ANY_CHECK)
+				return LZMA_GET_CHECK;
+		}
+
+	// Fall through
+
+	case SEQ_CODE: {
+		const lzma_ret ret = coder->next.code(
+				coder->next.coder, allocator,
+				in, in_pos, in_size,
+				out, out_pos, out_size, action);
+		if (ret != LZMA_STREAM_END
+				|| (coder->flags & LZMA_CONCATENATED) == 0)
 			return ret;
 
-		coder->initialized = true;
+		coder->sequence = SEQ_FINISH;
 	}
 
-	return coder->next.code(coder->next.coder, allocator,
-			in, in_pos, in_size, out, out_pos, out_size, action);
+	// Fall through
+
+	case SEQ_FINISH:
+		// When LZMA_DECODE_CONCATENATED was used and we were decoding
+		// LZMA_Alone file, we need to check check that there is no
+		// trailing garbage and wait for LZMA_FINISH.
+		if (*in_pos < in_size)
+			return LZMA_DATA_ERROR;
+
+		return action == LZMA_FINISH ? LZMA_STREAM_END : LZMA_OK;
+
+	default:
+		assert(0);
+		return LZMA_PROG_ERROR;
+	}
 }
 
 
 static void
 auto_decoder_end(lzma_coder *coder, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->next, allocator);
+	lzma_next_end(&coder->next, allocator);
 	lzma_free(coder, allocator);
 	return;
 }
 
 
+static lzma_check
+auto_decoder_get_check(const lzma_coder *coder)
+{
+	// It is LZMA_Alone if get_check is NULL.
+	return coder->next.get_check == NULL ? LZMA_CHECK_NONE
+			: coder->next.get_check(coder->next.coder);
+}
+
+
+static lzma_ret
+auto_decoder_memconfig(lzma_coder *coder, uint64_t *memusage,
+		uint64_t *old_memlimit, uint64_t new_memlimit)
+{
+	lzma_ret ret;
+
+	if (coder->next.memconfig != NULL) {
+		ret = coder->next.memconfig(coder->next.coder,
+				memusage, old_memlimit, new_memlimit);
+		assert(*old_memlimit == coder->memlimit);
+	} else {
+		// No coder is configured yet. Use the base value as
+		// the current memory usage.
+		*memusage = LZMA_MEMUSAGE_BASE;
+		*old_memlimit = coder->memlimit;
+		ret = LZMA_OK;
+	}
+
+	if (ret == LZMA_OK && new_memlimit != 0)
+		coder->memlimit = new_memlimit;
+
+	return ret;
+}
+
+
 static lzma_ret
 auto_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		lzma_extra **header, lzma_extra **footer)
+		uint64_t memlimit, uint32_t flags)
 {
+	lzma_next_coder_init(auto_decoder_init, next, allocator);
+
+	if (memlimit == 0)
+		return LZMA_PROG_ERROR;
+
+	if (flags & ~LZMA_SUPPORTED_FLAGS)
+		return LZMA_OPTIONS_ERROR;
+
 	if (next->coder == NULL) {
 		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
 		if (next->coder == NULL)
@@ -79,35 +168,26 @@ auto_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 
 		next->code = &auto_decode;
 		next->end = &auto_decoder_end;
+		next->get_check = &auto_decoder_get_check;
+		next->memconfig = &auto_decoder_memconfig;
 		next->coder->next = LZMA_NEXT_CODER_INIT;
 	}
 
-	next->coder->header = header;
-	next->coder->footer = footer;
-	next->coder->initialized = false;
+	next->coder->memlimit = memlimit;
+	next->coder->flags = flags;
+	next->coder->sequence = SEQ_INIT;
 
 	return LZMA_OK;
 }
 
 
-/*
-extern lzma_ret
-lzma_auto_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		lzma_extra **header, lzma_extra **footer)
-{
-	lzma_next_coder_init(
-			auto_decoder_init, next, allocator, header, footer);
-}
-*/
-
-
 extern LZMA_API lzma_ret
-lzma_auto_decoder(lzma_stream *strm, lzma_extra **header, lzma_extra **footer)
+lzma_auto_decoder(lzma_stream *strm, uint64_t memlimit, uint32_t flags)
 {
-	lzma_next_strm_init(strm, auto_decoder_init, header, footer);
+	lzma_next_strm_init(auto_decoder_init, strm, memlimit, flags);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
-	strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;
+	strm->internal->supported_actions[LZMA_FINISH] = true;
 
 	return LZMA_OK;
 }

src/liblzma/common/block_decoder.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_decoder.c
-/// \brief      Decodes .lzma Blocks
+/// \brief      Decodes .xz Blocks
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -18,119 +18,62 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #include "block_decoder.h"
-#include "block_private.h"
-#include "raw_decoder.h"
+#include "filter_decoder.h"
 #include "check.h"
 
 
 struct lzma_coder_s {
 	enum {
 		SEQ_CODE,
-		SEQ_CHECK,
-		SEQ_UNCOMPRESSED_SIZE,
-		SEQ_BACKWARD_SIZE,
 		SEQ_PADDING,
-		SEQ_END,
+		SEQ_CHECK,
 	} sequence;
 
 	/// The filters in the chain; initialized with lzma_raw_decoder_init().
 	lzma_next_coder next;
 
-	/// Decoding options; we also write Total Size, Compressed Size, and
-	/// Uncompressed Size back to this structure when the encoding has
-	/// been finished.
-	lzma_options_block *options;
+	/// Decoding options; we also write Compressed Size and Uncompressed
+	/// Size back to this structure when the decoding has been finished.
+	lzma_block *block;
 
-	/// Position in variable-length integers (and in some other places).
-	size_t pos;
-
-	/// Check of the uncompressed data
-	lzma_check check;
-
-	/// Total Size calculated while encoding
-	lzma_vli total_size;
-
-	/// Compressed Size calculated while encoding
+	/// Compressed Size calculated while decoding
 	lzma_vli compressed_size;
 
-	/// Uncompressed Size calculated while encoding
+	/// Uncompressed Size calculated while decoding
 	lzma_vli uncompressed_size;
 
-	/// Maximum allowed total_size
-	lzma_vli total_limit;
+	/// Maximum allowed Compressed Size; this takes into account the
+	/// size of the Block Header and Check fields when Compressed Size
+	/// is unknown.
+	lzma_vli compressed_limit;
 
-	/// Maximum allowed uncompressed_size
-	lzma_vli uncompressed_limit;
+	/// Position when reading the Check field
+	size_t check_pos;
 
-	/// Temporary location for the Uncompressed Size and Backward Size
-	/// fields in Block Footer.
-	lzma_vli tmp;
-
-	/// Size of the Backward Size field - This is needed so that we
-	/// can verify the Backward Size and still keep updating total_size.
-	size_t size_of_backward_size;
+	/// Check of the uncompressed data
+	lzma_check_state check;
 };
 
 
-static lzma_ret
-update_sequence(lzma_coder *coder)
+static inline bool
+update_size(lzma_vli *size, lzma_vli add, lzma_vli limit)
 {
-	switch (coder->sequence) {
-	case SEQ_CODE:
-		if (coder->options->check != LZMA_CHECK_NONE) {
-			lzma_check_finish(&coder->check,
-					coder->options->check);
-			coder->sequence = SEQ_CHECK;
-			break;
-		}
+	if (limit > LZMA_VLI_MAX)
+		limit = LZMA_VLI_MAX;
 
-	// Fall through
+	if (limit < *size || limit - *size < add)
+		return true;
 
-	case SEQ_CHECK:
-		if (coder->options->has_uncompressed_size_in_footer) {
-			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
-			break;
-		}
+	*size += add;
 
-	// Fall through
+	return false;
+}
 
-	case SEQ_UNCOMPRESSED_SIZE:
-		if (coder->options->has_backward_size) {
-			coder->sequence = SEQ_BACKWARD_SIZE;
-			break;
-		}
 
-	// Fall through
-
-	case SEQ_BACKWARD_SIZE:
-		if (coder->options->handle_padding) {
-			coder->sequence = SEQ_PADDING;
-			break;
-		}
-
-	case SEQ_PADDING:
-		if (!is_size_valid(coder->total_size,
-					coder->options->total_size)
-				|| !is_size_valid(coder->compressed_size,
-					coder->options->compressed_size)
-				|| !is_size_valid(coder->uncompressed_size,
-					coder->options->uncompressed_size))
-			return LZMA_DATA_ERROR;
-
-		// Copy the values into coder->options. The caller
-		// may use this information to construct Index.
-		coder->options->total_size = coder->total_size;
-		coder->options->compressed_size = coder->compressed_size;
-		coder->options->uncompressed_size = coder->uncompressed_size;
-
-		return LZMA_STREAM_END;
-
-	default:
-		assert(0);
-		return LZMA_PROG_ERROR;
-	}
-
-	return LZMA_OK;
+static inline bool
+is_size_valid(lzma_vli size, lzma_vli reference)
+{
+	return reference == LZMA_VLI_UNKNOWN || reference == size;
 }
 
 
@@ -140,166 +83,107 @@ block_decode(lzma_coder *coder, lzma_allocator *allocator,
 		size_t in_size, uint8_t *restrict out,
 		size_t *restrict out_pos, size_t out_size, lzma_action action)
 {
-	// Special case when the Block has only Block Header.
-	if (coder->sequence == SEQ_END)
-		return LZMA_STREAM_END;
-
-	// FIXME: Termination condition should work but could be cleaner.
-	while (*out_pos < out_size && (*in_pos < in_size
-			|| coder->sequence == SEQ_CODE))
 	switch (coder->sequence) {
 	case SEQ_CODE: {
 		const size_t in_start = *in_pos;
 		const size_t out_start = *out_pos;
 
-		lzma_ret ret = coder->next.code(coder->next.coder,
+		const lzma_ret ret = coder->next.code(coder->next.coder,
 				allocator, in, in_pos, in_size,
 				out, out_pos, out_size, action);
 
 		const size_t in_used = *in_pos - in_start;
 		const size_t out_used = *out_pos - out_start;
 
-		if (update_size(&coder->total_size, in_used,
-					coder->total_limit)
-				|| update_size(&coder->compressed_size,
-					in_used,
-					coder->options->compressed_size)
+		// NOTE: We compare to compressed_limit here, which prevents
+		// the total size of the Block growing past LZMA_VLI_MAX.
+		if (update_size(&coder->compressed_size, in_used,
+					coder->compressed_limit)
 				|| update_size(&coder->uncompressed_size,
-					out_used, coder->uncompressed_limit))
+					out_used,
+					coder->block->uncompressed_size))
 			return LZMA_DATA_ERROR;
 
-		lzma_check_update(&coder->check, coder->options->check,
+		lzma_check_update(&coder->check, coder->block->check,
 				out + out_start, out_used);
 
 		if (ret != LZMA_STREAM_END)
 			return ret;
 
-		ret = update_sequence(coder);
-		if (ret != LZMA_OK)
-			return ret;
+		// Compressed and Uncompressed Sizes are now at their final
+		// values. Verify that they match the values given to us.
+		if (!is_size_valid(coder->compressed_size,
+					coder->block->compressed_size)
+				|| !is_size_valid(coder->uncompressed_size,
+					coder->block->uncompressed_size))
+			return LZMA_DATA_ERROR;
 
-		break;
+		// Copy the values into coder->block. The caller
+		// may use this information to construct Index.
+		coder->block->compressed_size = coder->compressed_size;
+		coder->block->uncompressed_size = coder->uncompressed_size;
+
+		coder->sequence = SEQ_PADDING;
 	}
 
-	case SEQ_CHECK:
-		switch (coder->options->check) {
-		case LZMA_CHECK_CRC32:
-			if (((coder->check.crc32 >> (coder->pos * 8))
-					& 0xFF) != in[*in_pos])
-				return LZMA_DATA_ERROR;
-			break;
-
-		case LZMA_CHECK_CRC64:
-			if (((coder->check.crc64 >> (coder->pos * 8))
-					& 0xFF) != in[*in_pos])
-				return LZMA_DATA_ERROR;
-			break;
-
-		case LZMA_CHECK_SHA256:
-			if (coder->check.sha256.buffer[coder->pos]
-					!= in[*in_pos])
-				return LZMA_DATA_ERROR;
-			break;
-
-		default:
-			assert(coder->options->check != LZMA_CHECK_NONE);
-			assert(coder->options->check <= LZMA_CHECK_ID_MAX);
-			break;
-		}
-
-		if (update_size(&coder->total_size, 1, coder->total_limit))
-			return LZMA_DATA_ERROR;
-
-		++*in_pos;
-
-		if (++coder->pos == lzma_check_sizes[coder->options->check]) {
-			const lzma_ret ret = update_sequence(coder);
-			if (ret != LZMA_OK)
-				return ret;
-
-			coder->pos = 0;
-		}
-
-		break;
-
-	case SEQ_UNCOMPRESSED_SIZE: {
-		const size_t in_start = *in_pos;
-
-		lzma_ret ret = lzma_vli_decode(&coder->tmp,
-				&coder->pos, in, in_pos, in_size);
-
-		if (update_size(&coder->total_size, *in_pos - in_start,
-				coder->total_limit))
-			return LZMA_DATA_ERROR;
-
-		if (ret != LZMA_STREAM_END)
-			return ret;
-
-		if (coder->tmp != coder->uncompressed_size)
-			return LZMA_DATA_ERROR;
-
-		coder->pos = 0;
-		coder->tmp = 0;
-
-		ret = update_sequence(coder);
-		if (ret != LZMA_OK)
-			return ret;
-
-		break;
-	}
-
-	case SEQ_BACKWARD_SIZE: {
-		const size_t in_start = *in_pos;
-
-		lzma_ret ret = lzma_vli_decode(&coder->tmp,
-				&coder->pos, in, in_pos, in_size);
-
-		const size_t in_used = *in_pos - in_start;
-
-		if (update_size(&coder->total_size, in_used,
-				coder->total_limit))
-			return LZMA_DATA_ERROR;
-
-		coder->size_of_backward_size += in_used;
-
-		if (ret != LZMA_STREAM_END)
-			return ret;
-
-		if (coder->tmp != coder->total_size
-				- coder->size_of_backward_size)
-			return LZMA_DATA_ERROR;
-
-		ret = update_sequence(coder);
-		if (ret != LZMA_OK)
-			return ret;
-
-		break;
-	}
+	// Fall through
 
 	case SEQ_PADDING:
-		if (in[*in_pos] == 0x00) {
-			if (update_size(&coder->total_size, 1,
-					coder->total_limit))
-				return LZMA_DATA_ERROR;
+		// Compressed Data is padded to a multiple of four bytes.
+		while (coder->compressed_size & 3) {
+			// We use compressed_size here just get the Padding
+			// right. The actual Compressed Size was stored to
+			// coder->block already, and won't be modified by
+			// us anymore.
+			++coder->compressed_size;
 
-			++*in_pos;
-			break;
+			if (*in_pos >= in_size)
+				return LZMA_OK;
+
+			if (in[(*in_pos)++] != 0x00)
+				return LZMA_DATA_ERROR;
 		}
 
-		return update_sequence(coder);
+		if (coder->block->check == LZMA_CHECK_NONE)
+			return LZMA_STREAM_END;
 
-	default:
-		return LZMA_PROG_ERROR;
+		lzma_check_finish(&coder->check, coder->block->check);
+		coder->sequence = SEQ_CHECK;
+
+	// Fall through
+
+	case SEQ_CHECK: {
+		const bool chksup = lzma_check_is_supported(
+				coder->block->check);
+
+		while (*in_pos < in_size) {
+			// coder->check.buffer[] may be uninitialized when
+			// the Check ID is not supported.
+			if (chksup && coder->check.buffer.u8[coder->check_pos]
+					!= in[*in_pos]) {
+				++*in_pos;
+				return LZMA_DATA_ERROR;
+			}
+
+			++*in_pos;
+
+			if (++coder->check_pos == lzma_check_size(
+					coder->block->check))
+				return LZMA_STREAM_END;
+		}
+
+		return LZMA_OK;
+	}
 	}
 
-	return LZMA_OK;
+	return LZMA_PROG_ERROR;
 }
 
 
 static void
 block_decoder_end(lzma_coder *coder, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->next, allocator);
+	lzma_next_end(&coder->next, allocator);
 	lzma_free(coder, allocator);
 	return;
 }
@@ -307,11 +191,18 @@ block_decoder_end(lzma_coder *coder, lzma_allocator *allocator)
 
 extern lzma_ret
 lzma_block_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		lzma_options_block *options)
+		lzma_block *block)
 {
-	// This is pretty similar to lzma_block_encoder_init().
-	// See comments there.
+	lzma_next_coder_init(lzma_block_decoder_init, next, allocator);
 
+	// Validate the options. lzma_block_unpadded_size() does that for us
+	// except for Uncompressed Size and filters. Filters are validated
+	// by the raw decoder.
+	if (lzma_block_unpadded_size(block) == 0
+			|| !lzma_vli_is_valid(block->uncompressed_size))
+		return LZMA_PROG_ERROR;
+
+	// Allocate and initialize *next->coder if needed.
 	if (next->coder == NULL) {
 		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
 		if (next->coder == NULL)
@@ -322,74 +213,41 @@ lzma_block_decoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 		next->coder->next = LZMA_NEXT_CODER_INIT;
 	}
 
-	if (!lzma_vli_is_valid(options->total_size)
-			|| !lzma_vli_is_valid(options->compressed_size)
-			|| !lzma_vli_is_valid(options->uncompressed_size)
-			|| !lzma_vli_is_valid(options->total_size)
-			|| !lzma_vli_is_valid(options->total_limit)
-			|| !lzma_vli_is_valid(options->uncompressed_limit)
-			|| (options->uncompressed_size
-					!= LZMA_VLI_VALUE_UNKNOWN
-				&& options->uncompressed_size
-					> options->uncompressed_limit)
-			|| (options->total_size != LZMA_VLI_VALUE_UNKNOWN
-				&& options->total_size
-					> options->total_limit)
-			|| (!options->has_eopm && options->uncompressed_size
-				== LZMA_VLI_VALUE_UNKNOWN)
-			|| options->header_size > options->total_size
-			|| (options->handle_padding
-				&& (options->has_uncompressed_size_in_footer
-					|| options->has_backward_size)))
-		return LZMA_PROG_ERROR;
-
-	return_if_error(lzma_check_init(&next->coder->check, options->check));
-
-	if (!options->has_eopm && options->uncompressed_size == 0) {
-		if (!is_size_valid(0, options->compressed_size))
-			return LZMA_PROG_ERROR;
-
-		if (options->check != LZMA_CHECK_NONE) {
-			lzma_check_finish(&next->coder->check, options->check);
-			next->coder->sequence = SEQ_CHECK;
-		} else if (options->handle_padding) {
-			next->coder->sequence = SEQ_PADDING;
-		} else {
-			next->coder->sequence = SEQ_END;
-		}
-	} else {
-		next->coder->sequence = SEQ_CODE;
-	}
-
-	return_if_error(lzma_raw_decoder_init(&next->coder->next, allocator,
-			options->filters, options->has_eopm
-				? LZMA_VLI_VALUE_UNKNOWN
-				: options->uncompressed_size,
-			true));
-
-	next->coder->options = options;
-	next->coder->pos = 0;
-	next->coder->total_size = options->header_size;
+	// Basic initializations
+	next->coder->sequence = SEQ_CODE;
+	next->coder->block = block;
 	next->coder->compressed_size = 0;
 	next->coder->uncompressed_size = 0;
-	next->coder->total_limit
-			= MIN(options->total_size, options->total_limit);
-	next->coder->uncompressed_limit = MIN(options->uncompressed_size,
-			options->uncompressed_limit);
-	next->coder->tmp = 0;
-	next->coder->size_of_backward_size = 0;
 
-	return LZMA_OK;
+	// If Compressed Size is not known, we calculate the maximum allowed
+	// value so that encoded size of the Block (including Block Padding)
+	// is still a valid VLI and a multiple of four.
+	next->coder->compressed_limit
+			= block->compressed_size == LZMA_VLI_UNKNOWN
+				? (LZMA_VLI_MAX & ~LZMA_VLI_C(3))
+					- block->header_size
+					- lzma_check_size(block->check)
+				: block->compressed_size;
+
+	// Initialize the check. It's caller's problem if the Check ID is not
+	// supported, and the Block decoder cannot verify the Check field.
+	// Caller can test lzma_check_is_supported(block->check).
+	next->coder->check_pos = 0;
+	lzma_check_init(&next->coder->check, block->check);
+
+	// Initialize the filter chain.
+	return lzma_raw_decoder_init(&next->coder->next, allocator,
+			block->filters);
 }
 
 
 extern LZMA_API lzma_ret
-lzma_block_decoder(lzma_stream *strm, lzma_options_block *options)
+lzma_block_decoder(lzma_stream *strm, lzma_block *block)
 {
-	lzma_next_strm_init(strm, lzma_block_decoder_init, options);
+	lzma_next_strm_init(lzma_block_decoder_init, strm, block);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
-	strm->internal->supported_actions[LZMA_SYNC_FLUSH] = true;
+	strm->internal->supported_actions[LZMA_FINISH] = true;
 
 	return LZMA_OK;
 }

src/liblzma/common/block_decoder.h

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_decoder.h
-/// \brief      Decodes .lzma Blocks
+/// \brief      Decodes .xz Blocks
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -24,6 +24,6 @@
 
 
 extern lzma_ret lzma_block_decoder_init(lzma_next_coder *next,
-		lzma_allocator *allocator, lzma_options_block *options);
+		lzma_allocator *allocator, lzma_block *block);
 
 #endif

src/liblzma/common/block_encoder.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_encoder.c
-/// \brief      Encodes .lzma Blocks
+/// \brief      Encodes .xz Blocks
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -18,53 +18,51 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #include "block_encoder.h"
-#include "block_private.h"
-#include "raw_encoder.h"
+#include "filter_encoder.h"
 #include "check.h"
 
 
+/// The maximum size of a single Block is limited by the maximum size of
+/// a Stream, which is 2^63 - 1 bytes (i.e. LZMA_VLI_MAX). We could
+/// take into account the headers etc. to determine the exact maximum size
+/// of the Compressed Data field, but the complexity would give us nothing
+/// useful. Instead, limit the size of Compressed Data so that even with
+/// biggest possible Block Header and Check fields the total encoded size of
+/// the Block stays as valid VLI. This way we don't produce incorrect output
+/// if someone will really try creating a Block of 8 EiB.
+///
+/// ~LZMA_VLI_C(3) is to guarantee that if we need padding at the end of
+/// the Compressed Data field, it will still stay in the proper limit.
+#define COMPRESSED_SIZE_MAX ((LZMA_VLI_MAX - LZMA_BLOCK_HEADER_SIZE_MAX \
+		- LZMA_CHECK_SIZE_MAX) & ~LZMA_VLI_C(3))
+
+
 struct lzma_coder_s {
 	/// The filters in the chain; initialized with lzma_raw_decoder_init().
 	lzma_next_coder next;
 
-	/// Encoding options; we also write Total Size, Compressed Size, and
-	/// Uncompressed Size back to this structure when the encoding has
-	/// been finished.
-	lzma_options_block *options;
+	/// Encoding options; we also write Unpadded Size, Compressed Size,
+	/// and Uncompressed Size back to this structure when the encoding
+	/// has been finished.
+	lzma_block *block;
 
 	enum {
 		SEQ_CODE,
-		SEQ_CHECK_FINISH,
-		SEQ_CHECK_COPY,
-		SEQ_UNCOMPRESSED_SIZE,
-		SEQ_BACKWARD_SIZE,
 		SEQ_PADDING,
+		SEQ_CHECK,
 	} sequence;
 
-	/// Position in .header and .check.
-	size_t pos;
-
-	/// Check of the uncompressed data
-	lzma_check check;
-
-	/// Total Size calculated while encoding
-	lzma_vli total_size;
-
 	/// Compressed Size calculated while encoding
 	lzma_vli compressed_size;
 
 	/// Uncompressed Size calculated while encoding
 	lzma_vli uncompressed_size;
 
-	/// Maximum allowed total_size
-	lzma_vli total_limit;
+	/// Position in Block Padding and the Check fields
+	size_t pos;
 
-	/// Maximum allowed uncompressed_size
-	lzma_vli uncompressed_limit;
-
-	/// Backward Size - This is a copy of total_size right before
-	/// the Backward Size field.
-	lzma_vli backward_size;
+	/// Check of the uncompressed data
+	lzma_check_state check;
 };
 
 
@@ -75,26 +73,9 @@ block_encode(lzma_coder *coder, lzma_allocator *allocator,
 		size_t *restrict out_pos, size_t out_size, lzma_action action)
 {
 	// Check that our amount of input stays in proper limits.
-	if (coder->options->uncompressed_size != LZMA_VLI_VALUE_UNKNOWN) {
-		if (action == LZMA_FINISH) {
-			if (coder->options->uncompressed_size
-					- coder->uncompressed_size
-					!= (lzma_vli)(in_size - *in_pos))
-				return LZMA_DATA_ERROR;
-		} else {
-			if (coder->options->uncompressed_size
-					- coder->uncompressed_size
-					<  (lzma_vli)(in_size - *in_pos))
-				return LZMA_DATA_ERROR;
-		}
-	} else if (LZMA_VLI_VALUE_MAX - coder->uncompressed_size
-			< (lzma_vli)(in_size - *in_pos)) {
-		return LZMA_DATA_ERROR;
-	}
+	if (LZMA_VLI_MAX - coder->uncompressed_size < in_size - *in_pos)
+		return LZMA_PROG_ERROR;
 
-	// Main loop
-	while (*out_pos < out_size
-			&& (*in_pos < in_size || action == LZMA_FINISH))
 	switch (coder->sequence) {
 	case SEQ_CODE: {
 		const size_t in_start = *in_pos;
@@ -107,206 +88,101 @@ block_encode(lzma_coder *coder, lzma_allocator *allocator,
 		const size_t in_used = *in_pos - in_start;
 		const size_t out_used = *out_pos - out_start;
 
-		if (update_size(&coder->total_size, out_used,
-					coder->total_limit)
-				|| update_size(&coder->compressed_size,
-					out_used,
-					coder->options->compressed_size))
+		if (COMPRESSED_SIZE_MAX - coder->compressed_size < out_used)
 			return LZMA_DATA_ERROR;
 
+		coder->compressed_size += out_used;
+
 		// No need to check for overflow because we have already
 		// checked it at the beginning of this function.
 		coder->uncompressed_size += in_used;
 
-		lzma_check_update(&coder->check, coder->options->check,
+		lzma_check_update(&coder->check, coder->block->check,
 				in + in_start, in_used);
 
-		if (ret != LZMA_STREAM_END)
+		if (ret != LZMA_STREAM_END || action == LZMA_SYNC_FLUSH)
 			return ret;
 
 		assert(*in_pos == in_size);
+		assert(action == LZMA_FINISH);
 
-		// Compressed and Uncompressed Sizes are now at their final
-		// values. Verify that they match the values give to us.
-		if (!is_size_valid(coder->compressed_size,
-					coder->options->compressed_size)
-				|| !is_size_valid(coder->uncompressed_size,
-					coder->options->uncompressed_size))
-			return LZMA_DATA_ERROR;
+		// Copy the values into coder->block. The caller
+		// may use this information to construct Index.
+		coder->block->compressed_size = coder->compressed_size;
+		coder->block->uncompressed_size = coder->uncompressed_size;
 
-		coder->sequence = SEQ_CHECK_FINISH;
-		break;
+		coder->sequence = SEQ_PADDING;
 	}
 
-	case SEQ_CHECK_FINISH:
-		if (coder->options->check == LZMA_CHECK_NONE) {
-			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
-			break;
-		}
-
-		lzma_check_finish(&coder->check, coder->options->check);
-		coder->sequence = SEQ_CHECK_COPY;
-
 	// Fall through
 
-	case SEQ_CHECK_COPY:
-		assert(lzma_check_sizes[coder->options->check] > 0);
-
-		switch (coder->options->check) {
-		case LZMA_CHECK_CRC32:
-			out[*out_pos] = coder->check.crc32 >> (coder->pos * 8);
-			break;
-
-		case LZMA_CHECK_CRC64:
-			out[*out_pos] = coder->check.crc64 >> (coder->pos * 8);
-			break;
-
-		case LZMA_CHECK_SHA256:
-			out[*out_pos] = coder->check.sha256.buffer[coder->pos];
-			break;
-
-		default:
-			assert(0);
-			return LZMA_PROG_ERROR;
-		}
-
-		++*out_pos;
-
-		if (update_size(&coder->total_size, 1, coder->total_limit))
-			return LZMA_DATA_ERROR;
-
-		if (++coder->pos == lzma_check_sizes[coder->options->check]) {
-			coder->pos = 0;
-			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
-		}
-
-		break;
-
-	case SEQ_UNCOMPRESSED_SIZE:
-		if (coder->options->has_uncompressed_size_in_footer) {
-			const size_t out_start = *out_pos;
-
-			const lzma_ret ret = lzma_vli_encode(
-					coder->uncompressed_size,
-					&coder->pos, 1,
-					out, out_pos, out_size);
-
-			// Updating the size this way instead of doing in a
-			// single chunk using lzma_vli_size(), because this
-			// way we detect when exactly we are going out of
-			// our limits.
-			if (update_size(&coder->total_size,
-					*out_pos - out_start,
-					coder->total_limit))
-				return LZMA_DATA_ERROR;
-
-			if (ret != LZMA_STREAM_END)
-				return ret;
-
-			coder->pos = 0;
-		}
-
-		coder->backward_size = coder->total_size;
-		coder->sequence = SEQ_BACKWARD_SIZE;
-		break;
-
-	case SEQ_BACKWARD_SIZE:
-		if (coder->options->has_backward_size) {
-			const size_t out_start = *out_pos;
-
-			const lzma_ret ret = lzma_vli_encode(
-					coder->backward_size, &coder->pos, 1,
-					out, out_pos, out_size);
-
-			if (update_size(&coder->total_size,
-					*out_pos - out_start,
-					coder->total_limit))
-				return LZMA_DATA_ERROR;
-
-			if (ret != LZMA_STREAM_END)
-				return ret;
-		}
-
-		coder->sequence = SEQ_PADDING;
-		break;
-
 	case SEQ_PADDING:
-		if (coder->options->handle_padding) {
-			assert(!coder->options
-					->has_uncompressed_size_in_footer);
-			assert(!coder->options->has_backward_size);
-			assert(coder->options->total_size != LZMA_VLI_VALUE_UNKNOWN);
+		// Pad Compressed Data to a multiple of four bytes.
+		while ((coder->compressed_size + coder->pos) & 3) {
+			if (*out_pos >= out_size)
+				return LZMA_OK;
 
-			if (coder->total_size < coder->options->total_size) {
-				out[*out_pos] = 0x00;
-				++*out_pos;
-
-				if (update_size(&coder->total_size, 1,
-						coder->total_limit))
-					return LZMA_DATA_ERROR;
-
-				break;
-			}
+			out[*out_pos] = 0x00;
+			++*out_pos;
+			++coder->pos;
 		}
 
-		// Now also Total Size is known. Verify it.
-		if (!is_size_valid(coder->total_size,
-					coder->options->total_size))
-			return LZMA_DATA_ERROR;
+		if (coder->block->check == LZMA_CHECK_NONE)
+			return LZMA_STREAM_END;
 
-		// Copy the values into coder->options. The caller
-		// may use this information to construct Index.
-		coder->options->total_size = coder->total_size;
-		coder->options->compressed_size = coder->compressed_size;
-		coder->options->uncompressed_size = coder->uncompressed_size;
+		lzma_check_finish(&coder->check, coder->block->check);
 
-		return LZMA_STREAM_END;
+		coder->pos = 0;
+		coder->sequence = SEQ_CHECK;
 
-	default:
-		return LZMA_PROG_ERROR;
+	// Fall through
+
+	case SEQ_CHECK: {
+		const uint32_t check_size
+				= lzma_check_size(coder->block->check);
+
+		while (*out_pos < out_size) {
+			out[*out_pos] = coder->check.buffer.u8[coder->pos];
+			++*out_pos;
+
+			if (++coder->pos == check_size)
+				return LZMA_STREAM_END;
+		}
+
+		return LZMA_OK;
+	}
 	}
 
-	return LZMA_OK;
+	return LZMA_PROG_ERROR;
 }
 
 
 static void
 block_encoder_end(lzma_coder *coder, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->next, allocator);
+	lzma_next_end(&coder->next, allocator);
 	lzma_free(coder, allocator);
 	return;
 }
 
 
-static lzma_ret
-block_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		lzma_options_block *options)
+extern lzma_ret
+lzma_block_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
+		lzma_block *block)
 {
-	// Validate some options.
-	if (options == NULL
-			|| !lzma_vli_is_valid(options->total_size)
-			|| !lzma_vli_is_valid(options->compressed_size)
-			|| !lzma_vli_is_valid(options->uncompressed_size)
-			|| !lzma_vli_is_valid(options->total_size)
-			|| !lzma_vli_is_valid(options->total_limit)
-			|| !lzma_vli_is_valid(options->uncompressed_limit)
-			|| (options->uncompressed_size
-					!= LZMA_VLI_VALUE_UNKNOWN
-				&& options->uncompressed_size
-					> options->uncompressed_limit)
-			|| (options->total_size != LZMA_VLI_VALUE_UNKNOWN
-				&& options->total_size
-					> options->total_limit)
-			|| (!options->has_eopm && options->uncompressed_size
-				== LZMA_VLI_VALUE_UNKNOWN)
-			|| (options->handle_padding && (options->total_size
-					== LZMA_VLI_VALUE_UNKNOWN
-				|| options->has_uncompressed_size_in_footer
-				|| options->has_backward_size))
-			|| options->header_size > options->total_size)
+	lzma_next_coder_init(lzma_block_encoder_init, next, allocator);
+
+	if (block->version != 0)
+		return LZMA_OPTIONS_ERROR;
+
+	// If the Check ID is not supported, we cannot calculate the check and
+	// thus not create a proper Block.
+	if ((unsigned int)(block->check) > LZMA_CHECK_ID_MAX)
 		return LZMA_PROG_ERROR;
 
+	if (!lzma_check_is_supported(block->check))
+		return LZMA_UNSUPPORTED_CHECK;
+
 	// Allocate and initialize *next->coder if needed.
 	if (next->coder == NULL) {
 		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
@@ -318,55 +194,26 @@ block_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
 		next->coder->next = LZMA_NEXT_CODER_INIT;
 	}
 
-	// Initialize the check.
-	return_if_error(lzma_check_init(&next->coder->check, options->check));
-
-	// If End of Payload Marker is not used and Uncompressed Size is zero,
-	// Compressed Data is empty. That is, we don't call the encoder at all.
-	// We initialize it though; it allows detecting invalid options.
-	if (!options->has_eopm && options->uncompressed_size == 0) {
-		// Also Compressed Size must also be zero if it has been
-		// given to us.
-		if (!is_size_valid(options->compressed_size, 0))
-			return LZMA_PROG_ERROR;
-
-		next->coder->sequence = SEQ_CHECK_FINISH;
-	} else {
-		next->coder->sequence = SEQ_CODE;
-	}
-
-	// Other initializations
-	next->coder->options = options;
-	next->coder->pos = 0;
-	next->coder->total_size = options->header_size;
+	// Basic initializations
+	next->coder->sequence = SEQ_CODE;
+	next->coder->block = block;
 	next->coder->compressed_size = 0;
 	next->coder->uncompressed_size = 0;
-	next->coder->total_limit
-			= MIN(options->total_size, options->total_limit);
-	next->coder->uncompressed_limit = MIN(options->uncompressed_size,
-			options->uncompressed_limit);
+	next->coder->pos = 0;
+
+	// Initialize the check
+	lzma_check_init(&next->coder->check, block->check);
 
 	// Initialize the requested filters.
 	return lzma_raw_encoder_init(&next->coder->next, allocator,
-			options->filters, options->has_eopm
-				? LZMA_VLI_VALUE_UNKNOWN
-				: options->uncompressed_size,
-			true);
-}
-
-
-extern lzma_ret
-lzma_block_encoder_init(lzma_next_coder *next, lzma_allocator *allocator,
-		lzma_options_block *options)
-{
-	lzma_next_coder_init(block_encoder_init, next, allocator, options);
+			block->filters);
 }
 
 
 extern LZMA_API lzma_ret
-lzma_block_encoder(lzma_stream *strm, lzma_options_block *options)
+lzma_block_encoder(lzma_stream *strm, lzma_block *block)
 {
-	lzma_next_strm_init(strm, block_encoder_init, options);
+	lzma_next_strm_init(lzma_block_encoder_init, strm, block);
 
 	strm->internal->supported_actions[LZMA_RUN] = true;
 	strm->internal->supported_actions[LZMA_FINISH] = true;

src/liblzma/common/block_encoder.h

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_encoder.h
-/// \brief      Encodes .lzma Blocks
+/// \brief      Encodes .xz Blocks
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -24,6 +24,6 @@
 
 
 extern lzma_ret lzma_block_encoder_init(lzma_next_coder *next,
-		lzma_allocator *allocator, lzma_options_block *options);
+		lzma_allocator *allocator, lzma_block *block);
 
 #endif

src/liblzma/common/block_header_decoder.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_header_decoder.c
-/// \brief      Decodes Block Header from .lzma files
+/// \brief      Decodes Block Header from .xz files
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -21,353 +21,103 @@
 #include "check.h"
 
 
-struct lzma_coder_s {
-	lzma_options_block *options;
-
-	enum {
-		SEQ_FLAGS_1,
-		SEQ_FLAGS_2,
-		SEQ_COMPRESSED_SIZE,
-		SEQ_UNCOMPRESSED_SIZE,
-		SEQ_FILTER_FLAGS_INIT,
-		SEQ_FILTER_FLAGS_DECODE,
-		SEQ_CRC32,
-		SEQ_PADDING
-	} sequence;
-
-	/// Position in variable-length integers
-	size_t pos;
-
-	/// CRC32 of the Block Header
-	uint32_t crc32;
-
-	lzma_next_coder filter_flags_decoder;
-};
-
-
-static bool
-update_sequence(lzma_coder *coder)
-{
-	switch (coder->sequence) {
-	case SEQ_FLAGS_2:
-		if (coder->options->compressed_size
-				!= LZMA_VLI_VALUE_UNKNOWN) {
-			coder->pos = 0;
-			coder->sequence = SEQ_COMPRESSED_SIZE;
-			break;
-		}
-
-	// Fall through
-
-	case SEQ_COMPRESSED_SIZE:
-		if (coder->options->uncompressed_size
-				!= LZMA_VLI_VALUE_UNKNOWN) {
-			coder->pos = 0;
-			coder->sequence = SEQ_UNCOMPRESSED_SIZE;
-			break;
-		}
-
-	// Fall through
-
-	case SEQ_UNCOMPRESSED_SIZE:
-		coder->pos = 0;
-
-	// Fall through
-
-	case SEQ_FILTER_FLAGS_DECODE:
-		if (coder->options->filters[coder->pos].id
-				!= LZMA_VLI_VALUE_UNKNOWN) {
-			coder->sequence = SEQ_FILTER_FLAGS_INIT;
-			break;
-		}
-
-		if (coder->options->has_crc32) {
-			coder->pos = 0;
-			coder->sequence = SEQ_CRC32;
-			break;
-		}
-
-	case SEQ_CRC32:
-		if (coder->options->padding != 0) {
-			coder->pos = 0;
-			coder->sequence = SEQ_PADDING;
-			break;
-		}
-
-		return true;
-
-	default:
-		assert(0);
-		return true;
-	}
-
-	return false;
-}
-
-
-static lzma_ret
-block_header_decode(lzma_coder *coder, lzma_allocator *allocator,
-		const uint8_t *restrict in, size_t *restrict in_pos,
-		size_t in_size, uint8_t *restrict out lzma_attribute((unused)),
-		size_t *restrict out_pos lzma_attribute((unused)),
-		size_t out_size lzma_attribute((unused)),
-		lzma_action action lzma_attribute((unused)))
-{
-	while (*in_pos < in_size)
-	switch (coder->sequence) {
-	case SEQ_FLAGS_1:
-		// Check that the reserved bit is unset. Use HEADER_ERROR
-		// because newer version of liblzma may support the reserved
-		// bit, although it is likely that this is just a broken file.
-		if (in[*in_pos] & 0x40)
-			return LZMA_HEADER_ERROR;
-
-		// Number of filters: we prepare appropriate amount of
-		// variables for variable-length integer parsing. The
-		// initialization function has already reset the rest
-		// of the values to LZMA_VLI_VALUE_UNKNOWN, which allows
-		// us to later know how many filters there are.
-		for (int i = (int)(in[*in_pos] & 0x07) - 1; i >= 0; --i)
-			coder->options->filters[i].id = 0;
-
-		// End of Payload Marker flag
-		coder->options->has_eopm = (in[*in_pos] & 0x08) != 0;
-
-		// Compressed Size: Prepare for variable-length integer
-		// parsing if it is known.
-		if (in[*in_pos] & 0x10)
-			coder->options->compressed_size = 0;
-
-		// Uncompressed Size: the same.
-		if (in[*in_pos] & 0x20)
-			coder->options->uncompressed_size = 0;
-
-		// Is Metadata Block flag
-		coder->options->is_metadata = (in[*in_pos] & 0x80) != 0;
-
-		// We need at least one: Uncompressed Size or EOPM.
-		if (coder->options->uncompressed_size == LZMA_VLI_VALUE_UNKNOWN
-				&& !coder->options->has_eopm)
-			return LZMA_DATA_ERROR;
-
-		// Update header CRC32.
-		coder->crc32 = lzma_crc32(in + *in_pos, 1, coder->crc32);
-
-		++*in_pos;
-		coder->sequence = SEQ_FLAGS_2;
-		break;
-
-	case SEQ_FLAGS_2:
-		// Check that the reserved bits are unset.
-		if (in[*in_pos] & 0xE0)
-			return LZMA_DATA_ERROR;
-
-		// Get the size of Header Padding.
-		coder->options->padding = in[*in_pos] & 0x1F;
-
-		coder->crc32 = lzma_crc32(in + *in_pos, 1, coder->crc32);
-
-		++*in_pos;
-
-		if (update_sequence(coder))
-			return LZMA_STREAM_END;
-
-		break;
-
-	case SEQ_COMPRESSED_SIZE: {
-		// Store the old input position to be used when
-		// updating coder->header_crc32.
-		const size_t in_start = *in_pos;
-
-		const lzma_ret ret = lzma_vli_decode(
-				&coder->options->compressed_size,
-				&coder->pos, in, in_pos, in_size);
-
-		const size_t in_used = *in_pos - in_start;
-
-		coder->options->compressed_reserve += in_used;
-		assert(coder->options->compressed_reserve
-				<= LZMA_VLI_BYTES_MAX);
-
-		coder->options->header_size += in_used;
-
-		coder->crc32 = lzma_crc32(in + in_start, in_used,
-				coder->crc32);
-
-		if (ret != LZMA_STREAM_END)
-			return ret;
-
-		if (update_sequence(coder))
-			return LZMA_STREAM_END;
-
-		break;
-	}
-
-	case SEQ_UNCOMPRESSED_SIZE: {
-		const size_t in_start = *in_pos;
-
-		const lzma_ret ret = lzma_vli_decode(
-				&coder->options->uncompressed_size,
-				&coder->pos, in, in_pos, in_size);
-
-		const size_t in_used = *in_pos - in_start;
-
-		coder->options->uncompressed_reserve += in_used;
-		assert(coder->options->uncompressed_reserve
-				<= LZMA_VLI_BYTES_MAX);
-
-		coder->options->header_size += in_used;
-
-		coder->crc32 = lzma_crc32(in + in_start, in_used,
-				coder->crc32);
-
-		if (ret != LZMA_STREAM_END)
-			return ret;
-
-		if (update_sequence(coder))
-			return LZMA_STREAM_END;
-
-		break;
-	}
-
-	case SEQ_FILTER_FLAGS_INIT: {
-		assert(coder->options->filters[coder->pos].id
-				!= LZMA_VLI_VALUE_UNKNOWN);
-
-		const lzma_ret ret = lzma_filter_flags_decoder_init(
-				&coder->filter_flags_decoder, allocator,
-				&coder->options->filters[coder->pos]);
-		if (ret != LZMA_OK)
-			return ret;
-
-		coder->sequence = SEQ_FILTER_FLAGS_DECODE;
-	}
-
-	// Fall through
-
-	case SEQ_FILTER_FLAGS_DECODE: {
-		const size_t in_start = *in_pos;
-
-		const lzma_ret ret = coder->filter_flags_decoder.code(
-				coder->filter_flags_decoder.coder,
-				allocator, in, in_pos, in_size,
-				NULL, NULL, 0, LZMA_RUN);
-
-		const size_t in_used = *in_pos - in_start;
-		coder->options->header_size += in_used;
-		coder->crc32 = lzma_crc32(in + in_start,
-				in_used, coder->crc32);
-
-		if (ret != LZMA_STREAM_END)
-			return ret;
-
-		++coder->pos;
-
-		if (update_sequence(coder))
-			return LZMA_STREAM_END;
-
-		break;
-	}
-
-	case SEQ_CRC32:
-		assert(coder->options->has_crc32);
-
-		if (in[*in_pos] != ((coder->crc32 >> (coder->pos * 8)) & 0xFF))
-			return LZMA_DATA_ERROR;
-
-		++*in_pos;
-		++coder->pos;
-
-		// Check if we reached end of the CRC32 field.
-		if (coder->pos == 4) {
-			coder->options->header_size += 4;
-
-			if (update_sequence(coder))
-				return LZMA_STREAM_END;
-		}
-
-		break;
-
-	case SEQ_PADDING:
-		if (in[*in_pos] != 0x00)
-			return LZMA_DATA_ERROR;
-
-		++*in_pos;
-		++coder->options->header_size;
-		++coder->pos;
-
-		if (coder->pos < (size_t)(coder->options->padding))
-			break;
-
-		return LZMA_STREAM_END;
-
-	default:
-		return LZMA_PROG_ERROR;
-	}
-
-	return LZMA_OK;
-}
-
-
 static void
-block_header_decoder_end(lzma_coder *coder, lzma_allocator *allocator)
+free_properties(lzma_block *block, lzma_allocator *allocator)
 {
-	lzma_next_coder_end(&coder->filter_flags_decoder, allocator);
-	lzma_free(coder, allocator);
+	// Free allocated filter options. The last array member is not
+	// touched after the initialization in the beginning of
+	// lzma_block_header_decode(), so we don't need to touch that here.
+	for (size_t i = 0; i < LZMA_FILTERS_MAX; ++i) {
+		lzma_free(block->filters[i].options, allocator);
+		block->filters[i].id = LZMA_VLI_UNKNOWN;
+		block->filters[i].options = NULL;
+	}
+
 	return;
 }
 
 
-extern lzma_ret
-lzma_block_header_decoder_init(lzma_next_coder *next,
-		lzma_allocator *allocator, lzma_options_block *options)
-{
-	if (next->coder == NULL) {
-		next->coder = lzma_alloc(sizeof(lzma_coder), allocator);
-		if (next->coder == NULL)
-			return LZMA_MEM_ERROR;
-
-		next->code = &block_header_decode;
-		next->end = &block_header_decoder_end;
-		next->coder->filter_flags_decoder = LZMA_NEXT_CODER_INIT;
-	}
-
-	// Assume that Compressed Size and Uncompressed Size are unknown.
-	options->compressed_size = LZMA_VLI_VALUE_UNKNOWN;
-	options->uncompressed_size = LZMA_VLI_VALUE_UNKNOWN;
-
-	// We will calculate the sizes of these fields too so that the
-	// application may rewrite the header if it wishes so.
-	options->compressed_reserve = 0;
-	options->uncompressed_reserve = 0;
-
-	// The Block Flags field is always present, so include its size here
-	// and we don't need to worry about it in block_header_decode().
-	options->header_size = 2;
-
-	// Reset filters[] to indicate empty list of filters.
-	// See SEQ_FLAGS_1 in block_header_decode() for reasoning of this.
-	for (size_t i = 0; i < 8; ++i) {
-		options->filters[i].id = LZMA_VLI_VALUE_UNKNOWN;
-		options->filters[i].options = NULL;
-	}
-
-	next->coder->options = options;
-	next->coder->sequence = SEQ_FLAGS_1;
-	next->coder->pos = 0;
-	next->coder->crc32 = 0;
-
-	return LZMA_OK;
-}
-
-
 extern LZMA_API lzma_ret
-lzma_block_header_decoder(lzma_stream *strm,
-		lzma_options_block *options)
+lzma_block_header_decode(lzma_block *block,
+		lzma_allocator *allocator, const uint8_t *in)
 {
-	lzma_next_strm_init(strm, lzma_block_header_decoder_init, options);
+	// NOTE: We consider the header to be corrupt not only when the
+	// CRC32 doesn't match, but also when variable-length integers
+	// are invalid or over 63 bits, or if the header is too small
+	// to contain the claimed information.
 
-	strm->internal->supported_actions[LZMA_RUN] = true;
+	// Initialize the filter options array. This way the caller can
+	// safely free() the options even if an error occurs in this function.
+	for (size_t i = 0; i <= LZMA_FILTERS_MAX; ++i) {
+		block->filters[i].id = LZMA_VLI_UNKNOWN;
+		block->filters[i].options = NULL;
+	}
+
+	// Always zero for now.
+	block->version = 0;
+
+	// Validate Block Header Size and Check type. The caller must have
+	// already set these, so it is a programming error if this test fails.
+	if (lzma_block_header_size_decode(in[0]) != block->header_size
+			|| (unsigned int)(block->check) > LZMA_CHECK_ID_MAX)
+		return LZMA_PROG_ERROR;
+
+	// Exclude the CRC32 field.
+	const size_t in_size = block->header_size - 4;
+
+	// Verify CRC32
+	if (lzma_crc32(in, in_size, 0) != integer_read_32(in + in_size))
+		return LZMA_DATA_ERROR;
+
+	// Check for unsupported flags.
+	if (in[1] & 0x3C)
+		return LZMA_OPTIONS_ERROR;
+
+	// Start after the Block Header Size and Block Flags fields.
+	size_t in_pos = 2;
+
+	// Compressed Size
+	if (in[1] & 0x40) {
+		return_if_error(lzma_vli_decode(&block->compressed_size,
+				NULL, in, &in_pos, in_size));
+
+		// Validate Compressed Size. This checks that it isn't zero
+		// and that the total size of the Block is a valid VLI.
+		if (lzma_block_unpadded_size(block) == 0)
+			return LZMA_DATA_ERROR;
+	} else {
+		block->compressed_size = LZMA_VLI_UNKNOWN;
+	}
+
+	// Uncompressed Size
+	if (in[1] & 0x80)
+		return_if_error(lzma_vli_decode(&block->uncompressed_size,
+				NULL, in, &in_pos, in_size));
+	else
+		block->uncompressed_size = LZMA_VLI_UNKNOWN;
+
+	// Filter Flags
+	const size_t filter_count = (in[1] & 3) + 1;
+	for (size_t i = 0; i < filter_count; ++i) {
+		const lzma_ret ret = lzma_filter_flags_decode(
+				&block->filters[i], allocator,
+				in, &in_pos, in_size);
+		if (ret != LZMA_OK) {
+			free_properties(block, allocator);
+			return ret;
+		}
+	}
+
+	// Padding
+	while (in_pos < in_size) {
+		if (in[in_pos++] != 0x00) {
+			free_properties(block, allocator);
+
+			// Possibly some new field present so use
+			// LZMA_OPTIONS_ERROR instead of LZMA_DATA_ERROR.
+			return LZMA_OPTIONS_ERROR;
+		}
+	}
 
 	return LZMA_OK;
 }

src/liblzma/common/block_header_encoder.c

@@ -1,7 +1,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 //
 /// \file       block_header_encoder.c
-/// \brief      Encodes Block Header for .lzma files
+/// \brief      Encodes Block Header for .xz files
 //
 //  Copyright (C) 2007 Lasse Collin
 //
@@ -22,190 +22,118 @@
 
 
 extern LZMA_API lzma_ret
-lzma_block_header_size(lzma_options_block *options)
+lzma_block_header_size(lzma_block *block)
 {
-	// Block Flags take two bytes.
-	size_t size = 2;
+	if (block->version != 0)
+		return LZMA_OPTIONS_ERROR;
+
+	// Block Header Size + Block Flags + CRC32.
+	uint32_t size = 1 + 1 + 4;
 
 	// Compressed Size
-	if (!lzma_vli_is_valid(options->compressed_size)) {
-		return LZMA_PROG_ERROR;
-
-	} else if (options->compressed_reserve != 0) {
-		// Make sure that the known Compressed Size fits into the
-		// reserved space. Note that lzma_vli_size() will return zero
-		// if options->compressed_size is LZMA_VLI_VALUE_UNKNOWN, so
-		// we don't need to handle that special case separately.
-		if (options->compressed_reserve > LZMA_VLI_BYTES_MAX
-				|| lzma_vli_size(options->compressed_size)
-				> (size_t)(options->compressed_reserve))
+	if (block->compressed_size != LZMA_VLI_UNKNOWN) {
+		const uint32_t add = lzma_vli_size(block->compressed_size);
+		if (add == 0 || block->compressed_size == 0)
 			return LZMA_PROG_ERROR;
 
-		size += options->compressed_reserve;
-
-	} else if (options->compressed_size != LZMA_VLI_VALUE_UNKNOWN) {
-		// Compressed Size is known. We have already checked
-		// that is is a valid VLI, and since it isn't
-		// LZMA_VLI_VALUE_UNKNOWN, we can be sure that
-		// lzma_vli_size() will succeed.
-		size += lzma_vli_size(options->compressed_size);
+		size += add;
 	}
 
 	// Uncompressed Size
-	if (!lzma_vli_is_valid(options->uncompressed_size)) {
-		return LZMA_PROG_ERROR;
-
-	} else if (options->uncompressed_reserve != 0) {
-		if (options->uncompressed_reserve > LZMA_VLI_BYTES_MAX
-				|| lzma_vli_size(options->uncompressed_size)
-				> (size_t)(options->uncompressed_reserve))
+	if (block->uncompressed_size != LZMA_VLI_UNKNOWN) {
+		const uint32_t add = lzma_vli_size(block->uncompressed_size);
+		if (add == 0)
 			return LZMA_PROG_ERROR;
 
-		size += options->uncompressed_reserve;
-
-	} else if (options->uncompressed_size != LZMA_VLI_VALUE_UNKNOWN) {
-		size += lzma_vli_size(options->uncompressed_size);
+		size += add;
 	}
 
 	// List of Filter Flags
-	for (size_t i = 0; options->filters[i].id != LZMA_VLI_VALUE_UNKNOWN;
-			++i) {
+	if (block->filters == NULL || block->filters[0].id == LZMA_VLI_UNKNOWN)
+		return LZMA_PROG_ERROR;
+
+	for (size_t i = 0; block->filters[i].id != LZMA_VLI_UNKNOWN; ++i) {
 		// Don't allow too many filters.
-		if (i == 7)
+		if (i == LZMA_FILTERS_MAX)
 			return LZMA_PROG_ERROR;
 
-		uint32_t tmp;
-		const lzma_ret ret = lzma_filter_flags_size(&tmp,
-				options->filters + i);
-		if (ret != LZMA_OK)
-			return ret;
+		uint32_t add;
+		return_if_error(lzma_filter_flags_size(&add,
+				block->filters + i));
 
-		size += tmp;
+		size += add;
 	}
 
-	// CRC32
-	if (options->has_crc32)
-		size += 4;
+	// Pad to a multiple of four bytes.
+	block->header_size = (size + 3) & ~UINT32_C(3);
 
-	// Padding
-	int32_t padding;
-	if (options->padding == LZMA_BLOCK_HEADER_PADDING_AUTO) {
-		const uint32_t preferred = lzma_alignment_output(
-				options->filters, 1);
-		const uint32_t unaligned = size + options->alignment;
-		padding = (int32_t)(unaligned % preferred);
-		if (padding != 0)
-			padding = preferred - padding;
-	} else if (options->padding >= LZMA_BLOCK_HEADER_PADDING_MIN
-			&& options->padding <= LZMA_BLOCK_HEADER_PADDING_MAX) {
-		padding = options->padding;
-	} else {
-		return LZMA_PROG_ERROR;
-	}
-
-	// All success. Copy the calculated values to the options structure.
-	options->padding = padding;
-	options->header_size = size + (size_t)(padding);
+	// NOTE: We don't verify that the encoded size of the Block stays
+	// within limits. This is because it is possible that we are called
+	// with exaggerated Compressed Size (e.g. LZMA_VLI_MAX) to reserve
+	// space for Block Header, and later called again with lower,
+	// real values.
 
 	return LZMA_OK;
 }
 
 
 extern LZMA_API lzma_ret
-lzma_block_header_encode(uint8_t *out, const lzma_options_block *options)
+lzma_block_header_encode(const lzma_block *block, uint8_t *out)
 {
-	// We write the Block Flags later.
-	if (options->header_size < 2)
+	// Valdidate everything but filters.
+	if (lzma_block_unpadded_size(block) == 0
+			|| !lzma_vli_is_valid(block->uncompressed_size))
 		return LZMA_PROG_ERROR;
 
-	const size_t out_size = options->header_size;
+	// Indicate the size of the buffer _excluding_ the CRC32 field.
+	const size_t out_size = block->header_size - 4;
+
+	// Store the Block Header Size.
+	out[0] = out_size / 4;
+
+	// We write Block Flags in pieces.
+	out[1] = 0x00;
 	size_t out_pos = 2;
 
 	// Compressed Size
-	if (options->compressed_size != LZMA_VLI_VALUE_UNKNOWN
-			|| options->compressed_reserve != 0) {
-		const lzma_vli size = options->compressed_size
-					!= LZMA_VLI_VALUE_UNKNOWN
-				? options->compressed_size : 0;
-		size_t vli_pos = 0;
-		if (lzma_vli_encode(
-				size, &vli_pos, options->compressed_reserve,
-				out, &out_pos, out_size) != LZMA_STREAM_END)
-			return LZMA_PROG_ERROR;
+	if (block->compressed_size != LZMA_VLI_UNKNOWN) {
+		return_if_error(lzma_vli_encode(block->compressed_size, NULL,
+				out, &out_pos, out_size));
 
+		out[1] |= 0x40;
 	}
 
 	// Uncompressed Size
-	if (options->uncompressed_size != LZMA_VLI_VALUE_UNKNOWN
-			|| options->uncompressed_reserve != 0) {
-		const lzma_vli size = options->uncompressed_size
-					!= LZMA_VLI_VALUE_UNKNOWN
-				? options->uncompressed_size : 0;
-		size_t vli_pos = 0;
-		if (lzma_vli_encode(
-				size, &vli_pos, options->uncompressed_reserve,
-				out, &out_pos, out_size) != LZMA_STREAM_END)
-			return LZMA_PROG_ERROR;
+	if (block->uncompressed_size != LZMA_VLI_UNKNOWN) {
+		return_if_error(lzma_vli_encode(block->uncompressed_size, NULL,
+				out, &out_pos, out_size));
 
+		out[1] |= 0x80;
 	}
 
 	// Filter Flags
-	size_t filter_count;
-	for (filter_count = 0; options->filters[filter_count].id
-			!= LZMA_VLI_VALUE_UNKNOWN; ++filter_count) {
-		// There can be at maximum of seven filters.
-		if (filter_count == 7)
+	if (block->filters == NULL || block->filters[0].id == LZMA_VLI_UNKNOWN)
+		return LZMA_PROG_ERROR;
+
+	size_t filter_count = 0;
+	do {
+		// There can be at maximum of four filters.
+		if (filter_count == LZMA_FILTERS_MAX)
 			return LZMA_PROG_ERROR;
 
-		const lzma_ret ret = lzma_filter_flags_encode(out, &out_pos,
-				out_size, options->filters + filter_count);
-		// FIXME: Don't return LZMA_BUF_ERROR.
-		if (ret != LZMA_OK)
-			return ret;
-	}
+		return_if_error(lzma_filter_flags_encode(
+				block->filters + filter_count,
+				out, &out_pos, out_size));
 
-	// Block Flags 1
-	out[0] = filter_count;
+	} while (block->filters[++filter_count].id != LZMA_VLI_UNKNOWN);
 
-	if (options->has_eopm)
-		out[0] |= 0x08;
-	else if (options->uncompressed_size == LZMA_VLI_VALUE_UNKNOWN)
-		return LZMA_PROG_ERROR;
+	out[1] |= filter_count - 1;
 
-	if (options->compressed_size != LZMA_VLI_VALUE_UNKNOWN
-			|| options->compressed_reserve != 0)
-		out[0] |= 0x10;
-
-	if (options->uncompressed_size != LZMA_VLI_VALUE_UNKNOWN
-			|| options->uncompressed_reserve != 0)
-		out[0] |= 0x20;
-
-	if (options->is_metadata)
-		out[0] |= 0x80;
-
-	// Block Flags 2
-	if (options->padding < LZMA_BLOCK_HEADER_PADDING_MIN
-			|| options->padding > LZMA_BLOCK_HEADER_PADDING_MAX)
-		return LZMA_PROG_ERROR;
-
-	out[1] = (uint8_t)(options->padding);
+	// Padding
+	memzero(out + out_pos, out_size - out_pos);
 
 	// CRC32
-	if (options->has_crc32) {
-		if (out_size - out_pos < 4)
-			return LZMA_PROG_ERROR;
-
-		const uint32_t crc = lzma_crc32(out, out_pos, 0);
-		for (size_t i = 0; i < 4; ++i)
-			out[out_pos++] = crc >> (i * 8);
-	}
-
-	// Padding - the amount of available space must now match with
-	// the size of the Padding field.
-	if (out_size - out_pos != (size_t)(options->padding))
-		return LZMA_PROG_ERROR;
-
-	memzero(out + out_pos, (size_t)(options->padding));
+	integer_write_32(out + out_size, lzma_crc32(out, out_size, 0));
 
 	return LZMA_OK;
 }

src/liblzma/common/block_util.c

@@ -0,0 +1,97 @@
+///////////////////////////////////////////////////////////////////////////////
+//
+/// \file       block_header.c
+/// \brief      Utility functions to handle lzma_block
+//
+//  Copyright (C) 2008 Lasse Collin
+//
+//  This library is free software; you can redistribute it and/or
+//  modify it under the terms of the GNU Lesser General Public
+//  License as published by the Free Software Foundation; either
+//  version 2.1 of the License, or (at your option) any later version.
+//
+//  This library is distributed in the hope that it will be useful,
+//  but WITHOUT ANY WARRANTY; without even the implied warranty of
+//  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+//  Lesser General Public License for more details.
+//
+///////////////////////////////////////////////////////////////////////////////
+
+#include "common.h"
+#include "index.h"
+
+
+extern LZMA_API lzma_ret
+lzma_block_compressed_size(lzma_block *block, lzma_vli total_size)
+{
+	// Validate everything but Uncompressed Size and filters.
+	if (lzma_block_unpadded_size(block) == 0)
+		return LZMA_PROG_ERROR;
+
+	const uint32_t container_size = block->header_size
+			+ lzma_check_size(block->check);
+
+	// Validate that Compressed Size will be greater than zero.
+	if (container_size <= total_size)
+		return LZMA_DATA_ERROR;
+
+	// Calculate what Compressed Size is supposed to be.
+	// If Compressed Size was present in Block Header,
+	// compare that the new value matches it.
+	const lzma_vli compressed_size = total_size - container_size;
+	if (block->compressed_size != LZMA_VLI_UNKNOWN
+			&& block->compressed_size != compressed_size)
+		return LZMA_DATA_ERROR;
+
+	block->compressed_size = compressed_size;
+
+	return LZMA_OK;
+}
+
+
+extern LZMA_API lzma_vli
+lzma_block_unpadded_size(const lzma_block *block)
+{
+	// Validate the values that we are interested in i.e. all but
+	// Uncompressed Size and the filters.
+	//
+	// NOTE: This function is used for validation too, so it is
+	// essential that these checks are always done even if
+	// Compressed Size is unknown.
+	if (block->version != 0
+			|| block->header_size < LZMA_BLOCK_HEADER_SIZE_MIN
+			|| block->header_size > LZMA_BLOCK_HEADER_SIZE_MAX
+			|| (block->header_size & 3)
+			|| !lzma_vli_is_valid(block->compressed_size)
+			|| block->compressed_size == 0
+			|| (unsigned int)(block->check) > LZMA_CHECK_ID_MAX)
+		return 0;
+
+	// If Compressed Size is unknown, return that we cannot know
+	// size of the Block either.
+	if (block->compressed_size == LZMA_VLI_UNKNOWN)
+		return LZMA_VLI_UNKNOWN;
+
+	// Calculate Unpadded Size and validate it.
+	const lzma_vli unpadded_size = block->compressed_size
+				+ block->header_size
+				+ lzma_check_size(block->check);
+
+	assert(unpadded_size >= UNPADDED_SIZE_MIN);
+	if (unpadded_size > UNPADDED_SIZE_MAX)
+		return 0;
+
+	return unpadded_size;
+}
+
+
+extern LZMA_API lzma_vli
+lzma_block_total_size(const lzma_block *block)
+{
+	lzma_vli unpadded_size = lzma_block_unpadded_size(block);
+
+	if (unpadded_size != LZMA_VLI_UNKNOWN)
+		unpadded_size = vli_ceil4(unpadded_size);
+
+	return unpadded_size;
+}