OISF/suricata
1
0
Fork 0
mirror of https://github.com/OISF/suricata.git synced 2026-10-11 22:09:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

http: Use libhtp-rs.

Browse Source 
Ticket: #2696

There are a lot of changes here, which are described below.

In general these changes are renaming constants to conform to the
libhtp-rs versions (which are generated by cbindgen); making all htp
types opaque and changing struct->member references to
htp_struct_member() function calls; and a handful of changes to offload
functionality onto libhtp-rs from suricata, such as URI normalization
and transaction cleanup.

Functions introduced to handle opaque htp_tx_t:
- tx->parsed_uri => htp_tx_parsed_uri(tx)
- tx->parsed_uri->path => htp_uri_path(htp_tx_parsed_uri(tx)
- tx->parsed_uri->hostname => htp_uri_hostname(htp_tx_parsed_uri(tx))
- htp_tx_get_user_data() => htp_tx_user_data(tx)
- htp_tx_is_http_2_upgrade(tx) convenience function introduced to detect response status 101
  and “Upgrade: h2c" header.

Functions introduced to handle opaque htp_tx_data_t:
- d->len => htp_tx_data_len()
- d->data => htp_tx_data_data()
- htp_tx_data_tx(data) function to get the htp_tx_t from the htp_tx_data_t
- htp_tx_data_is_empty(data) convenience function introduced to test if the data is empty.

Other changes:

Build libhtp-rs as a crate inside rust. Update autoconf to no longer
use libhtp as an external dependency. Remove HAVE_HTP feature defines
since they are no longer needed.

Make function arguments and return values const where possible

htp_tx_destroy(tx) will now free an incomplete transaction

htp_time_t replaced with standard struct timeval

Callbacks from libhtp now provide the htp_connp_t and the htp_tx_data_t
as separate arguments. This means the connection parser is no longer
fetched from the transaction inside callbacks.

SCHTPGenerateNormalizedUri() functionality moved inside libhtp-rs, which
now provides normalized URI values.
The normalized URI is available with accessor function: htp_tx_normalized_uri()
Configuration settings added to control the behaviour of the URI normalization:
- htp_config_set_normalized_uri_include_all()
- htp_config_set_plusspace_decode()
- htp_config_set_convert_lowercase()
- htp_config_set_double_decode_normalized_query()
- htp_config_set_double_decode_normalized_path()
- htp_config_set_backslash_convert_slashes()
- htp_config_set_bestfit_replacement_byte()
- htp_config_set_convert_lowercase()
- htp_config_set_nul_encoded_terminates()
- htp_config_set_nul_raw_terminates()
- htp_config_set_path_separators_compress()
- htp_config_set_path_separators_decode()
- htp_config_set_u_encoding_decode()
- htp_config_set_url_encoding_invalid_handling()
- htp_config_set_utf8_convert_bestfit()
- htp_config_set_normalized_uri_include_all()
- htp_config_set_plusspace_decode()
Constants related to configuring uri normalization:
- HTP_URL_DECODE_PRESERVE_PERCENT => HTP_URL_ENCODING_HANDLING_PRESERVE_PERCENT
- HTP_URL_DECODE_REMOVE_PERCENT => HTP_URL_ENCODING_HANDLING_REMOVE_PERCENT
- HTP_URL_DECODE_PROCESS_INVALID => HTP_URL_ENCODING_HANDLING_PROCESS_INVALID

htp_config_set_field_limits(soft_limit, hard_limit) changed to
htp_config_set_field_limit(limit) because libhtp didn't implement soft
limits.

libhtp logging API updated to provide HTP_LOG_CODE constants along with
the message. This eliminates the need to perform string matching on
message text to map log messages to HTTP_DECODER_EVENT values, and the
HTP_LOG_CODE values can be used directly. In support of this,
HTP_DECODER_EVENT values are mapped to their corresponding HTP_LOG_CODE
values.

New log events to describe additional anomalies:
HTP_LOG_CODE_REQUEST_TOO_MANY_LZMA_LAYERS
HTP_LOG_CODE_RESPONSE_TOO_MANY_LZMA_LAYERS
HTP_LOG_CODE_PROTOCOL_CONTAINS_EXTRA_DATA
HTP_LOG_CODE_CONTENT_LENGTH_EXTRA_DATA_START
HTP_LOG_CODE_CONTENT_LENGTH_EXTRA_DATA_END
HTP_LOG_CODE_SWITCHING_PROTO_WITH_CONTENT_LENGTH
HTP_LOG_CODE_DEFORMED_EOL
HTP_LOG_CODE_PARSER_STATE_ERROR
HTP_LOG_CODE_MISSING_OUTBOUND_TRANSACTION_DATA
HTP_LOG_CODE_MISSING_INBOUND_TRANSACTION_DATA
HTP_LOG_CODE_ZERO_LENGTH_DATA_CHUNKS
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD_NO_PROTOCOL
HTP_LOG_CODE_REQUEST_LINE_UNKNOWN_METHOD_INVALID_PROTOCOL
HTP_LOG_CODE_REQUEST_LINE_NO_PROTOCOL
HTP_LOG_CODE_RESPONSE_LINE_INVALID_PROTOCOL
HTP_LOG_CODE_RESPONSE_LINE_INVALID_RESPONSE_STATUS
HTP_LOG_CODE_RESPONSE_BODY_INTERNAL_ERROR
HTP_LOG_CODE_REQUEST_BODY_DATA_CALLBACK_ERROR
HTP_LOG_CODE_RESPONSE_INVALID_EMPTY_NAME
HTP_LOG_CODE_REQUEST_INVALID_EMPTY_NAME
HTP_LOG_CODE_RESPONSE_INVALID_LWS_AFTER_NAME
HTP_LOG_CODE_RESPONSE_HEADER_NAME_NOT_TOKEN
HTP_LOG_CODE_REQUEST_INVALID_LWS_AFTER_NAME
HTP_LOG_CODE_LZMA_DECOMPRESSION_DISABLED
HTP_LOG_CODE_CONNECTION_ALREADY_OPEN
HTP_LOG_CODE_COMPRESSION_BOMB_DOUBLE_LZMA
HTP_LOG_CODE_INVALID_CONTENT_ENCODING
HTP_LOG_CODE_INVALID_GAP
HTP_LOG_CODE_ERROR

The new htp_log API supports consuming log messages more easily than
walking a list and tracking the current offset. Internally, libhtp-rs
now provides log messages as a queue of htp_log_t, which means the
application can simply call htp_conn_next_log() to fetch the next log
message until the queue is empty. Once the application is done with a
log message, they can call htp_log_free() to dispose of it.

Functions supporting htp_log_t:
htp_conn_next_log(conn) - Get the next log message
htp_log_message(log) - To get the text of the message
htp_log_code(log) - To get the HTP_LOG_CODE value
htp_log_free(log) - To free the htp_log_t
This commit is contained in:
Todd Mortimer
2025-04-04 02:35:12 +02:00
committed by Victor Julien
parent 9409a5a49a
commit 9c324b796e
258 changed files with 24015 additions and 1385 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/PULL_REQUEST_TEMPLATE.md
+1 -3
View File
@@ -24,7 +24,7 @@ Describe changes:
### Provide values to any of the below to override the defaults.
- To use an LibHTP, Suricata-Verify or Suricata-Update pull request,
- To use a Suricata-Verify or Suricata-Update pull request,
link to the pull request in the respective `_BRANCH` variable.
- Leave unused overrides blank or remove.
@@ -32,5 +32,3 @@ SV_REPO=
SV_BRANCH=
SU_REPO=
SU_BRANCH=
LIBHTP_REPO=
LIBHTP_BRANCH=
.github/workflows/builds.yml
+3 -112
View File
@@ -13,8 +13,6 @@ on:
- "doc/**"
workflow_dispatch:
inputs:
LIBHTP_REPO:
LIBHTP_BRANCH:
SU_REPO:
SU_BRANCH:
SV_REPO:
@@ -145,7 +143,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xvf prep/libhtp.tar.gz
- run: tar xvf prep/suricata-update.tar.gz
- run: tar xvf prep/suricata-verify.tar.gz
- name: Configuring
@@ -293,7 +290,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xvf prep/libhtp.tar.gz
- run: tar xvf prep/suricata-update.tar.gz
- run: tar xvf prep/suricata-verify.tar.gz
- name: Build
@@ -311,74 +307,6 @@ jobs:
run: cargo clippy --all-features
working-directory: rust
almalinux-9-non-bundled-libhtp:
name: AlmaLinux 9 Non-Bundled LibHTP
runs-on: ubuntu-latest
container: almalinux:9
needs: [prepare-deps, ubuntu-22-04-dist]
steps:
# Cache Rust stuff.
- name: Cache cargo registry
uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c30a57
with:
path: ~/.cargo/registry
key: cargo-registry
- name: Determine number of CPUs
run: echo CPUS=$(nproc --all) >> $GITHUB_ENV
- name: Install system packages
run: |
dnf -y install dnf-plugins-core epel-release
dnf config-manager --set-enabled crb
dnf -y install \
autoconf \
automake \
cargo-vendor \
cbindgen \
diffutils \
numactl-devel \
dpdk-devel \
file-devel \
gcc \
gcc-c++ \
git \
jansson-devel \
jq \
libtool \
libyaml-devel \
libnfnetlink-devel \
libnetfilter_queue-devel \
libnet-devel \
libcap-ng-devel \
libevent-devel \
libmaxminddb-devel \
libpcap-devel \
libtool \
lz4-devel \
make \
pcre2-devel \
pkgconfig \
python3-devel \
python3-sphinx \
python3-yaml \
rust-toolset \
sudo \
which \
zlib-devel
- name: Download suricata.tar.gz
uses: actions/download-artifact@95815c38cf2ff2164869cbab79da8d1f422bc89e
with:
name: dist
- run: tar xf suricata-*.tar.gz --strip-components=1
- run: cd libhtp && ./configure --prefix=/usr/local
- run: cd libhtp && make -j ${{ env.CPUS }}
- run: cd libhtp && make install
- run: PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --enable-non-bundled-htp --with-libhtp-includes=/usr/local/include --with-libhtp-libraries=/usr/local/lib
rpms:
name: Build RPMs
runs-on: ubuntu-latest
@@ -391,7 +319,7 @@ jobs:
- almalinux:9
- fedora:41
env:
skip: false
skip: true
steps:
- name: Cache cargo registry
uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c30a57
@@ -526,7 +454,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xvf prep/libhtp.tar.gz
- run: tar xvf prep/suricata-update.tar.gz
- run: tar xvf prep/suricata-verify.tar.gz
- uses: ./.github/actions/install-cbindgen
@@ -704,7 +631,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: ./autogen.sh
- run: ./configure --enable-warnings --disable-shared
@@ -793,7 +719,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: ./autogen.sh
- run: CC="clang" CFLAGS="$DEFAULT_CFLAGS -Wshadow" ./configure --disable-shared
@@ -887,7 +812,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: ./autogen.sh
- run: ./configure --enable-warnings --enable-debug --enable-unittests --disable-shared --enable-rust-strict --enable-hiredis --enable-nfqueue
@@ -973,7 +897,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- run: mkdir /home/suricata/suricata
@@ -1058,7 +981,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: ./autogen.sh
- run: |
if ./configure; then
@@ -1113,7 +1035,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure
- run: make -j ${{ env.CPUS }}
@@ -1181,7 +1102,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- run: ./autogen.sh
@@ -1258,7 +1178,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: ./configure --enable-warnings --disable-shared --enable-unittests
@@ -1314,14 +1233,6 @@ jobs:
CARGO_INCREMENTAL: 0
- run: llvm-profdata-14 merge -o ct.profdata /tmp/ct.profraw
- run: llvm-cov-14 show $(find rust/target/debug/deps/ -type f -regex 'rust/target/debug/deps/suricata\-[a-z0-9]+$') -instr-profile=ct.profdata --show-instantiations --ignore-filename-regex="^/root/.*" >> coverage.txt
- run: |
cd libhtp
make test
cd ..
env:
LLVM_PROFILE_FILE: "/tmp/htp-test.profraw"
- run: llvm-profdata-14 merge -o htp-test.profdata /tmp/htp-test.profraw
- run: llvm-cov-14 show libhtp/test/test_all -instr-profile=htp-test.profdata --show-instantiations --ignore-filename-regex="^/root/.*" >> coverage.txt
- name: Upload coverage to Codecov
uses: codecov/codecov-action@0565863a31f2c772f9f0395002a31e3f06189574
with:
@@ -1399,7 +1310,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- uses: ./.github/actions/install-cbindgen
- name: Fix kernel mmap rnd bits
@@ -1505,7 +1415,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- name: Fix kernel mmap rnd bits
# Asan in llvm 14 provided in ubuntu 22.04 is incompatible with
@@ -1640,7 +1549,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- name: Extracting suricata-verify
run: tar xf prep/suricata-verify.tar.gz
- name: Fix kernel mmap rnd bits
@@ -1728,7 +1636,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- name: Fix kernel mmap rnd bits
run: sudo sysctl vm.mmap_rnd_bits=28
- run: ./autogen.sh
@@ -1831,7 +1738,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- name: Fix kernel mmap rnd bits
# Asan in llvm 14 provided in ubuntu 22.04 is incompatible with
@@ -1922,7 +1828,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
@@ -2072,7 +1977,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- name: Fix kernel mmap rnd bits
# Asan in llvm 14 provided in ubuntu 22.04 is incompatible with
@@ -2150,7 +2054,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: AFL_HARDEN=1 ac_cv_func_realloc_0_nonnull=yes ac_cv_func_malloc_0_nonnull=yes CFLAGS="-fsanitize=address -fno-omit-frame-pointer" CXXFLAGS=$CFLAGS CC=afl-clang-fast CXX=afl-clang-fast++ LDFLAGS="-fsanitize=address" ./configure --enable-warnings --enable-fuzztargets --disable-shared
@@ -2246,7 +2149,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-netmap
@@ -2281,7 +2183,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- run: ./autogen.sh
@@ -2382,7 +2283,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-dpdk
@@ -2463,7 +2363,6 @@ jobs:
run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain $(grep rust-version rust/Cargo.toml.in|sed 's/\"//g'|awk '{print $3}') -y
- run: echo "$HOME/.cargo/bin" >> $GITHUB_PATH
- uses: ./.github/actions/install-cbindgen
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- run: ./autogen.sh
@@ -2548,7 +2447,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
@@ -2634,7 +2532,6 @@ jobs:
run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain $(grep rust-version rust/Cargo.toml.in|sed 's/\"//g'|awk '{print $3}') -y
- run: echo "$HOME/.cargo/bin" >> $GITHUB_PATH
- uses: ./.github/actions/install-cbindgen
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: tar xf prep/suricata-verify.tar.gz
- run: ./autogen.sh
@@ -2710,7 +2607,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
@@ -2781,7 +2677,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- uses: ./.github/actions/install-cbindgen
- run: ./autogen.sh
@@ -2832,7 +2727,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xvf prep/libhtp.tar.gz
- run: tar xvf prep/suricata-update.tar.gz
- name: Create Python virtual environment
run: python3 -m venv ./testenv
@@ -2843,14 +2737,13 @@ jobs:
- run: ./autogen.sh
- run: CPATH="$HOMEBREW_PREFIX/include:$CPATH" LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LIBRARY_PATH" PATH="/opt/homebrew/opt/libtool/libexec/gnubin:$PATH" CFLAGS="${DEFAULT_CFLAGS}" ./configure --enable-warnings --enable-unittests --prefix="$HOME/.local/"
- run: CPATH="$HOMEBREW_PREFIX/include:$CPATH" LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LIBRARY_PATH" PATH="/opt/homebrew/opt/libtool/libexec/gnubin:$PATH" CFLAGS="${DEFAULT_CFLAGS}" make -j2
# somehow it gets included by some C++ stdlib header (case unsensitive)
- run: rm libhtp/VERSION && make check
- run: CPATH="$HOMEBREW_PREFIX/include:$CPATH" LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LIBRARY_PATH" PATH="/opt/homebrew/opt/libtool/libexec/gnubin:$PATH" CFLAGS="${DEFAULT_CFLAGS}" make check
- run: tar xf prep/suricata-verify.tar.gz
- name: Running suricata-verify
run: |
. ./testenv/bin/activate
python3 ./suricata-verify/run.py -q --debug-failed
- run: make install
- run: CPATH="$HOMEBREW_PREFIX/include:$CPATH" LIBRARY_PATH="$HOMEBREW_PREFIX/lib:$LIBRARY_PATH" PATH="/opt/homebrew/opt/libtool/libexec/gnubin:$PATH" CFLAGS="${DEFAULT_CFLAGS}" make install
- name: Check Suricata-Update
run: |
. ./testenv/bin/activate
@@ -2886,7 +2779,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- name: Npcap DLL
run: |
@@ -2989,7 +2881,6 @@ jobs:
with:
name: prep
path: prep
- run: tar xf prep/libhtp.tar.gz
- name: WinDivert
run: |
curl -sL -O https://github.com/basil00/Divert/releases/download/v1.4.3/WinDivert-1.4.3-A.zip
.github/workflows/codeql.yml
-1
View File
@@ -54,7 +54,6 @@ jobs:
sudo apt-get install libjansson-dev
sudo apt-get install libpcap-dev
sudo apt-get install libnuma-dev
git clone --depth 1 https://github.com/OISF/libhtp.git
cargo install cbindgen
export PATH=/opt/work/.cargo/bin:$PATH
chmod +x autogen.sh
.github/workflows/commits.yml
-1
View File
@@ -80,7 +80,6 @@ jobs:
# The action above is supposed to do this for us, but it doesn't appear to stick.
- run: /usr/bin/git config --global --add safe.directory /__w/suricata/suricata
- run: git fetch
- run: git clone https://github.com/OISF/libhtp -b 0.5.x
- name: Building all commits
run: |
echo "Building commits from ${GITHUB_BASE_REF}."
.github/workflows/docs.yml
-3
View File
@@ -8,8 +8,6 @@ on:
pull_request:
workflow_dispatch:
inputs:
LIBHTP_REPO:
LIBHTP_BRANCH:
SU_REPO:
SU_BRANCH:
SV_REPO:
@@ -130,7 +128,6 @@ jobs:
cp prep/cbindgen $HOME/.cargo/bin
chmod 755 $HOME/.cargo/bin/cbindgen
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
- run: tar xf prep/libhtp.tar.gz
- run: tar xf prep/suricata-update.tar.gz
- run: ./autogen.sh
- run: CFLAGS="${DEFAULT_CFLAGS}" ./configure
.github/workflows/formatting.yml
-1
View File
@@ -128,7 +128,6 @@ jobs:
echo "No github merge commit found"
fi
shell: bash {0}
- run: git clone https://github.com/OISF/libhtp -b 0.5.x
- run: ./autogen.sh
- run: ./configure --enable-warnings --enable-unittests
- name: Check formatting
.github/workflows/prepare-deps.yml
-18
View File
@@ -29,9 +29,6 @@ jobs:
echo "Parsing branch and PR info from:"
echo "${body}"
LIBHTP_REPO=$(echo "${body}" | awk -F = '/^LIBHTP_REPO=/ { print $2 }')
LIBHTP_BRANCH=$(echo "${body}" | awk -F = '/^LIBHTP_BRANCH=/ { print $2 }')
SU_REPO=$(echo "${body}" | awk -F = '/^SU_REPO=/ { print $2 }')
SU_BRANCH=$(echo "${body}" | awk -F = '/^SU_BRANCH=/ { print $2 }')
@@ -39,8 +36,6 @@ jobs:
SV_BRANCH=$(echo "${body}" | awk -F = '/^SV_BRANCH=/ { print $2 }')
else
echo "No pull request body, will use inputs or defaults."
LIBHTP_REPO=${{ inputs.LIBHTP_REPO }}
LIBHTP_BRANCH=${{ inputs.LIBHTP_BRANCH }}
SU_REPO=${{ inputs.SU_REPO }}
SU_BRANCH=${{ inputs.SU_BRANCH }}
SV_REPO=${{ inputs.SV_REPO }}
@@ -48,9 +43,6 @@ jobs:
fi
# If the _REPO variables don't contain a full URL, add GitHub.
if [ "${LIBHTP_REPO}" ] && ! echo "${LIBHTP_REPO}" | grep -q '^https://'; then
LIBHTP_REPO="https://github.com/${LIBHTP_REPO}"
fi
if [ "${SU_REPO}" ] && ! echo "${SU_REPO}" | grep -q '^https://'; then
SU_REPO="https://github.com/${SU_REPO}"
fi
@@ -58,9 +50,6 @@ jobs:
SV_REPO="https://github.com/${SV_REPO}"
fi
echo LIBHTP_REPO=${LIBHTP_REPO} | tee -a ${GITHUB_ENV}
echo LIBHTP_BRANCH=${LIBHTP_BRANCH} | tee -a ${GITHUB_ENV}
echo SU_REPO=${SU_REPO} | tee -a ${GITHUB_ENV}
echo SU_BRANCH=${SU_BRANCH} | tee -a ${GITHUB_ENV}
@@ -69,8 +58,6 @@ jobs:
- name: Annotate output
run: |
echo "::notice:: LIBHTP_REPO=${LIBHTP_REPO}"
echo "::notice:: LIBHTP_BRANCH=${LIBHTP_BRANCH}"
echo "::notice:: SU_REPO=${SU_REPO}"
echo "::notice:: SU_BRANCH=${SU_BRANCH}"
echo "::notice:: SV_REPO=${SV_REPO}"
@@ -81,10 +68,6 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- run: git config --global --add safe.directory /__w/suricata/suricata
- name: Fetching libhtp
run: |
DESTDIR=./bundle ./scripts/bundle.sh libhtp
tar zcf libhtp.tar.gz -C bundle libhtp
- name: Fetching suricata-update
run: |
DESTDIR=./bundle ./scripts/bundle.sh suricata-update
@@ -116,6 +99,5 @@ jobs:
with:
name: prep
path: |
libhtp.tar.gz
suricata-update.tar.gz
suricata-verify.tar.gz
.github/workflows/rust-checks.yml
-2
View File
@@ -76,7 +76,6 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Configure Suricata
run: |
./scripts/bundle.sh libhtp
./autogen.sh
./configure --enable-warnings
- name: Run Cargo Audit
@@ -151,7 +150,6 @@ jobs:
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
- name: Configure Suricata
run: |
./scripts/bundle.sh libhtp
./autogen.sh
./configure --enable-warnings
- name: Cargo Update and Build
.github/workflows/rust.yml
+3
View File
@@ -107,3 +107,6 @@ jobs:
echo "::error ::Cargo.lock.in needs to be updated"
exit 1
fi
# does not work in other subdirectories for now
- run: cargo fmt
working-directory: rust/htp
.github/workflows/scan-build.yml
+1 -2
View File
@@ -74,11 +74,10 @@ jobs:
- run: scan-build-18 ./configure --enable-warnings --enable-dpdk --enable-nfqueue --enable-nflog
env:
CC: clang-18
# exclude libhtp from the analysis
# disable security.insecureAPI.DeprecatedOrUnsafeBufferHandling explicitly as
# this will require significant effort to address.
- run: |
scan-build-18 --status-bugs --exclude libhtp/ --exclude rust \
scan-build-18 --status-bugs --exclude rust \
-enable-checker valist.Uninitialized \
-enable-checker valist.CopyToSelf \
-enable-checker valist.Unterminated \
.gitignore
-13
View File
@@ -32,19 +32,6 @@ doc/userguide/suricata.1
etc/suricata.logrotate
etc/suricata.service
install-sh
libhtp/TAGS
libhtp/aclocal.m4
libhtp/autom4te.cache/
libhtp/config.h
libhtp/config.log
libhtp/config.status
libhtp/configure
libhtp/htp.pc
libhtp/htp/TAGS
libhtp/htp/libhtp.la
libhtp/libtool
libhtp/stamp-h1
libhtp/test/TAGS
libtool
ltmain.sh
missing
Makefile.am
+2 -2
View File
@@ -12,7 +12,7 @@ EXTRA_DIST = ChangeLog COPYING LICENSE suricata.yaml.in \
scripts/docs-ubuntu-debian-minimal-build.sh \
scripts/evedoc.py \
examples/plugins
SUBDIRS = $(HTP_DIR) rust src plugins qa rules doc etc python ebpf \
SUBDIRS = rust src plugins qa rules doc etc python ebpf \
$(SURICATA_UPDATE_DIR)
DIST_SUBDIRS = $(SUBDIRS) examples/lib/simple examples/lib/custom
@@ -54,7 +54,7 @@ endif
@echo "You can now start suricata by running as root something like:"
@echo " $(DESTDIR)$(bindir)/suricata -c $(DESTDIR)$(e_sysconfdir)suricata.yaml -i eth0"
@echo ""
@echo "If a library like libhtp.so is not found, you can run suricata with:"
@echo "If a shared library is not found, you can add library paths with:"
@echo " LD_LIBRARY_PATH="$(DESTDIR)$(prefix)/lib" "$(DESTDIR)$(bindir)/suricata" -c "$(DESTDIR)$(e_sysconfdir)suricata.yaml" -i eth0"
@echo ""
@echo "The Emerging Threats Open rules are now installed. Rules can be"
configure.ac
-104
View File
@@ -1562,108 +1562,6 @@
[test "x$install_suricata_update" = "xyes"])
AC_SUBST([install_suricata_update_reason])
# libhtp
AC_ARG_ENABLE(non-bundled-htp,
AS_HELP_STRING([--enable-non-bundled-htp], [Enable the use of an already installed version of htp]),[enable_non_bundled_htp=$enableval],[enable_non_bundled_htp=no])
AS_IF([test "x$enable_non_bundled_htp" = "xyes"], [
PKG_CHECK_MODULES([libhtp], htp,, [with_pkgconfig_htp=no])
if test "$with_pkgconfig_htp" != "no"; then
CPPFLAGS="${CPPFLAGS} ${libhtp_CFLAGS}"
LIBS="${LIBS} ${libhtp_LIBS}"
fi
AC_ARG_WITH(libhtp_includes,
[ --with-libhtp-includes=DIR libhtp include directory],
[with_libhtp_includes="$withval"],[with_libhtp_includes=no])
AC_ARG_WITH(libhtp_libraries,
[ --with-libhtp-libraries=DIR libhtp library directory],
[with_libhtp_libraries="$withval"],[with_libhtp_libraries="no"])
if test "$with_libhtp_includes" != "no"; then
CPPFLAGS="-I${with_libhtp_includes} ${CPPFLAGS}"
fi
if test "$with_libhtp_libraries" != "no"; then
LDFLAGS="${LDFLAGS} -L${with_libhtp_libraries}"
fi
AC_CHECK_HEADER(htp/htp.h,,[AC_MSG_ERROR(htp/htp.h not found ...)])
LIBHTP=""
AC_CHECK_LIB(htp, htp_conn_create,, LIBHTP="no")
if test "$LIBHTP" = "no"; then
echo
echo " ERROR! libhtp library not found"
echo
exit 1
fi
PKG_CHECK_MODULES(LIBHTPMINVERSION, [htp >= 0.5.45],[libhtp_minver_found="yes"],[libhtp_minver_found="no"])
if test "$libhtp_minver_found" = "no"; then
PKG_CHECK_MODULES(LIBHTPDEVVERSION, [htp = 0.5.X],[libhtp_devver_found="yes"],[libhtp_devver_found="no"])
if test "$libhtp_devver_found" = "no"; then
echo
echo " ERROR! libhtp was found but it is neither >= 0.5.45, nor the dev 0.5.X"
echo
exit 1
fi
fi
AC_CHECK_LIB([htp], [htp_config_register_request_uri_normalize],AC_DEFINE_UNQUOTED([HAVE_HTP_URI_NORMALIZE_HOOK],[1],[Found htp_config_register_request_uri_normalize function in libhtp]) ,,[-lhtp])
# check for htp_tx_get_response_headers_raw
AC_CHECK_LIB([htp], [htp_tx_get_response_headers_raw],AC_DEFINE_UNQUOTED([HAVE_HTP_TX_GET_RESPONSE_HEADERS_RAW],[1],[Found htp_tx_get_response_headers_raw in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_decode_query_inplace],AC_DEFINE_UNQUOTED([HAVE_HTP_DECODE_QUERY_INPLACE],[1],[Found htp_decode_query_inplace function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_response_decompression_layer_limit],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_RESPONSE_DECOMPRESSION_LAYER_LIMIT],[1],[Found htp_config_set_response_decompression_layer_limit function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_allow_space_uri],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_ALLOW_SPACE_URI],[1],[Found htp_config_set_allow_space_uri function in libhtp]) ,,[-lhtp])
AC_EGREP_HEADER(htp_config_set_path_decode_u_encoding, htp/htp.h, AC_DEFINE_UNQUOTED([HAVE_HTP_SET_PATH_DECODE_U_ENCODING],[1],[Found usable htp_config_set_path_decode_u_encoding function in libhtp]) )
AC_CHECK_LIB([htp], [htp_config_set_lzma_memlimit],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_LZMA_MEMLIMIT],[1],[Found htp_config_set_lzma_memlimit function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_lzma_layers],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_LZMA_LAYERS],[1],[Found htp_config_set_lzma_layers function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_compression_bomb_limit],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_COMPRESSION_BOMB_LIMIT],[1],[Found htp_config_set_compression_bomb_limit function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_compression_time_limit],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_COMPRESSION_TIME_LIMIT],[1],[Found htp_config_set_compression_time_limit function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_max_tx],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_MAX_TX],[1],[Found htp_config_set_max_tx function in libhtp]) ,,[-lhtp])
AC_CHECK_LIB([htp], [htp_config_set_number_headers_limit],AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_HEADERS_LIMIT],[1],[Found htp_config_set_number_headers_limit function in libhtp]) ,,[-lhtp])
])
if test "x$enable_non_bundled_htp" = "xno"; then
# test if we have a bundled htp
if test -d "$srcdir/libhtp"; then
AC_CONFIG_SUBDIRS([libhtp])
HTP_DIR="libhtp"
AC_SUBST(HTP_DIR)
HTP_LDADD="../libhtp/htp/libhtp.la"
AC_SUBST(HTP_LDADD)
# make sure libhtp is added to the includes
CPPFLAGS="-I\${srcdir}/../libhtp/ ${CPPFLAGS}"
AC_CHECK_HEADER(iconv.h,,[AC_MSG_ERROR(iconv.h not found ...)])
AC_CHECK_LIB(iconv, libiconv_close)
AC_DEFINE_UNQUOTED([HAVE_HTP_URI_NORMALIZE_HOOK],[1],[Assuming htp_config_register_request_uri_normalize function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_TX_GET_RESPONSE_HEADERS_RAW],[1],[Assuming htp_tx_get_response_headers_raw function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_DECODE_QUERY_INPLACE],[1],[Assuming htp_decode_query_inplace function in bundled libhtp])
# enable when libhtp has been updated
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_RESPONSE_DECOMPRESSION_LAYER_LIMIT],[1],[Assuming htp_config_set_response_decompression_layer_limit function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_ALLOW_SPACE_URI],[1],[Assuming htp_config_set_allow_space_uri function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_LZMA_MEMLIMIT],[1],[Assuming htp_config_set_lzma_memlimit function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_LZMA_LAYERS],[1],[Assuming htp_config_set_lzma_layers function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_COMPRESSION_BOMB_LIMIT],[1],[Assuming htp_config_set_compression_bomb_limit function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_COMPRESSION_TIME_LIMIT],[1],[Assuming htp_config_set_compression_time_limit function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_MAX_TX],[1],[Assuming htp_config_set_max_tx function in bundled libhtp])
AC_DEFINE_UNQUOTED([HAVE_HTP_CONFIG_SET_HEADERS_LIMIT],[1],[Assuming htp_config_set_number_headers_limit function in bundled libhtp])
else
echo
echo " ERROR: Libhtp is not bundled. Get libhtp by doing:"
echo " git clone https://github.com/OISF/libhtp"
echo " Then re-run Suricata's autogen.sh and configure script."
echo " Or, if libhtp is installed in a different location,"
echo " pass --enable-non-bundled-htp to Suricata's configure script."
echo " Add --with-libhtp-includes=<dir> and --with-libhtp-libraries=<dir> if"
echo " libhtp is not installed in the include and library paths."
echo
exit 1
fi
fi
AM_CONDITIONAL([HTP_LDADD], [test "x${HTP_LDADD}" != "x"])
# Check for libcap-ng
case $host in
*-*-linux*)
@@ -2582,7 +2480,6 @@ AC_SUBST(MAJOR_MINOR)
AC_SUBST(RUST_FEATURES)
AC_SUBST(RUST_SURICATA_LIBDIR)
AC_SUBST(RUST_SURICATA_LIBNAME)
AC_SUBST(enable_non_bundled_htp)
AM_CONDITIONAL([BUILD_SHARED_LIBRARY], [test "x$enable_shared" = "xyes"] && [test "x$can_build_shared_library" = "xyes"])
@@ -2634,7 +2531,6 @@ SURICATA_BUILD_CONF="Suricata Configuration:
GeoIP2 support: ${enable_geoip}
JA3 support: ${enable_ja3}
JA4 support: ${enable_ja4}
Non-bundled htp: ${enable_non_bundled_htp}
Hyperscan support: ${enable_hyperscan}
Libnet support: ${enable_libnet}
liblz4 support: ${enable_liblz4}
doc/userguide/devguide/codebase/installation-from-git.rst
+1 -1
View File
@@ -72,7 +72,7 @@ Next, enter the following line in the terminal:
git clone https://github.com/OISF/suricata.git
cd suricata
Libhtp and suricata-update are not bundled. Get them by doing:
Suricata-update is not bundled. Get it by doing:
.. code-block:: bash
doc/userguide/devguide/codebase/testing.rst
+1 -1
View File
@@ -30,7 +30,7 @@ Use these to check that specific functions behave as expected, in success and in
during development, for nom parsers in the Rust codebase, for instance, or for checking that messages
or message parts of a protocol/stream are processed as they should.
To execute all unit tests (both from C and Rust code), as well as ``libhtp`` ones, from the Suricata main directory, run::
To execute all unit tests (both from C and Rust code) from the Suricata main directory, run::
make check
doc/userguide/lua/lua-functions.rst
+1 -1
View File
@@ -103,7 +103,7 @@ Example:
HttpGetRequestHost
~~~~~~~~~~~~~~~~~~
Get the host from libhtp's tx->request_hostname, which can either be
Get the host from libhtp's htp_tx_request_hostname(tx), which can either be
the host portion of the url or the host portion of the Host header.
Example:
doc/userguide/upgrade.rst
+4
View File
@@ -171,6 +171,10 @@ Logging changes
live transaction, to reduce the chances of logging unrelated data.** This may
lead to what looks like a regression in behavior, but it is a considered choice.
Other Changes
~~~~~~~~~~~~~
- libhtp has been replaced with a rust version. This means libhtp is no longer built and linked as a shared library, and the libhtp dependency is now built directly into suricata.
Upgrading 6.0 to 7.0
--------------------
doxygen.cfg
+1 -1
View File
@@ -829,7 +829,7 @@ WARN_LOGFILE =
# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING
# Note: If this tag is empty the current directory is searched.
INPUT = src/ libhtp/htp/ examples/
INPUT = src/ examples/
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
examples/lib/custom/Makefile.am
-3
View File
@@ -6,7 +6,4 @@ AM_CPPFLAGS = -I$(top_srcdir)/src
custom_LDFLAGS = $(all_libraries) $(SECLDFLAGS)
custom_LDADD = "-Wl,--start-group,$(top_builddir)/src/libsuricata_c.a,../../$(RUST_SURICATA_LIB),--end-group" $(RUST_LDADD)
if HTP_LDADD
custom_LDADD += ../../$(HTP_LDADD)
endif
custom_DEPENDENCIES = $(top_builddir)/src/libsuricata_c.a ../../$(RUST_SURICATA_LIB)
examples/lib/simple/Makefile.am
-3
View File
@@ -6,7 +6,4 @@ AM_CPPFLAGS = -I$(top_srcdir)/src
simple_LDFLAGS = $(all_libraries) $(SECLDFLAGS)
simple_LDADD = "-Wl,--start-group,$(top_builddir)/src/libsuricata_c.a,../../$(RUST_SURICATA_LIB),--end-group" $(RUST_LDADD)
if HTP_LDADD
simple_LDADD += ../../$(HTP_LDADD)
endif
simple_DEPENDENCIES = $(top_builddir)/src/libsuricata_c.a ../../$(RUST_SURICATA_LIB)
libsuricata-config.in
-8
View File
@@ -9,8 +9,6 @@ LIBS="@LIBS@ @RUST_LDADD@"
shared_lib="-lsuricata"
static_lib="-lsuricata_c -lsuricata_rust"
enable_non_bundled_htp="@enable_non_bundled_htp@"
lib="$shared_lib"
show_libs="no"
@@ -47,12 +45,6 @@ if [ "$use_static" = "no" ]; then
fi
fi
# If we're using a bundled htp, add it to the libs as well. It will
# already be present if we're use a non-bundled libhtp.
if [ "$enable_non_bundled_htp" = "no" ]; then
lib="${lib} -lhtp"
fi
output=""
if [ "$show_cflags" = "yes" ]; then
requirements.txt
+1 -2
View File
@@ -1,7 +1,6 @@
# Specify libhtp and suricata-update requirements.
# Specify suricata-update requirements.
#
# Format:
#
# name {repo} {branch|tag}
libhtp https://github.com/OISF/libhtp 0.5.x
suricata-update https://github.com/OISF/suricata-update master
rules/http-events.rules
+2 -1
View File
@@ -96,4 +96,5 @@ alert http any any -> any any (msg:"SURICATA HTTP request missing protocol"; flo
alert http any any -> any any (msg:"SURICATA HTTP request too many headers"; flow:established,to_server; app-layer-event:http.request_too_many_headers; classtype:protocol-command-decode; sid:2221056; rev:1;)
alert http any any -> any any (msg:"SURICATA HTTP response too many headers"; flow:established,to_client; app-layer-event:http.response_too_many_headers; classtype:protocol-command-decode; sid:2221057; rev:1;)
# next sid 2221058
#alert http any any -> any any (msg:"SURICATA HTTP response chunk extension"; flow:established; app-layer-event:http.response_chunk_extension; classtype:protocol-command-decode; sid:2221058; rev:1;)
# next sid 2221059
rust/Cargo.lock.in
+235 -4
View File
@@ -175,6 +175,17 @@ dependencies = [
"alloc-stdlib",
]
[[package]]
name = "bstr"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6798148dccfbff0fae41c7574d2fa8f1ef3492fba0face179de5d8d447d67b05"
dependencies = [
"memchr",
"regex-automata",
"serde",
]
[[package]]
name = "build_const"
version = "0.2.2"
@@ -187,6 +198,21 @@ version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
[[package]]
name = "cc"
version = "1.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c736e259eea577f443d5c86c304f9f4ae0295c43f3ba05c21f1d66b5f06001af"
dependencies = [
"shlex",
]
[[package]]
name = "cdylib-link-lines"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d98eabef08bbdf5afd0b9c0cabb1ac335f7c70447ef095eed85dffd9628b20bc"
[[package]]
name = "cfg-if"
version = "1.0.0"
@@ -450,6 +476,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c936bfdafb507ebbf50b8074c54fa31c5be9a1e7e5f467dd659697041407d07c"
dependencies = [
"crc32fast",
"libz-sys",
"miniz_oxide",
]
@@ -465,6 +492,101 @@ version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
[[package]]
name = "futures"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65bc07b1a8bc7c85c5f2e110c476c7389b4554ba72af57d8445ea63a576b0876"
dependencies = [
"futures-channel",
"futures-core",
"futures-executor",
"futures-io",
"futures-sink",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-channel"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
dependencies = [
"futures-core",
"futures-sink",
]
[[package]]
name = "futures-core"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e"
[[package]]
name = "futures-executor"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e28d1d997f585e54aebc3f97d39e72338912123a67330d723fdbb564d646c9f"
dependencies = [
"futures-core",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-io"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6"
[[package]]
name = "futures-macro"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.98",
]
[[package]]
name = "futures-sink"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7"
[[package]]
name = "futures-task"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"
[[package]]
name = "futures-timer"
version = "3.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f288b0a4f20f9a56b5d1da57e2227c661b7b16168e2f72365f57b63326e29b24"
[[package]]
name = "futures-util"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81"
dependencies = [
"futures-channel",
"futures-core",
"futures-io",
"futures-macro",
"futures-sink",
"futures-task",
"memchr",
"pin-project-lite",
"pin-utils",
"slab",
]
[[package]]
name = "generic-array"
version = "0.14.7"
@@ -537,6 +659,22 @@ dependencies = [
"digest",
]
[[package]]
name = "htp"
version = "2.0.0"
dependencies = [
"base64",
"bstr",
"cdylib-link-lines",
"flate2",
"lazy_static",
"libc",
"lzma-rs",
"nom",
"rstest",
"time",
]
[[package]]
name = "indexmap"
version = "2.7.1"
@@ -607,6 +745,18 @@ dependencies = [
"libc",
]
[[package]]
name = "libz-sys"
version = "1.1.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df9b68e50e6e0b26f672573834882eb57759f6db9b3be2ea3c35c91188bb4eaa"
dependencies = [
"cc",
"libc",
"pkg-config",
"vcpkg",
]
[[package]]
name = "linux-raw-sys"
version = "0.4.15"
@@ -945,6 +1095,18 @@ version = "0.2.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b"
[[package]]
name = "pin-utils"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "pkg-config"
version = "0.3.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "953ec861398dccce10c670dfeaf3ec4911ca479e9c02154b3a215178c5f566f2"
[[package]]
name = "polyval"
version = "0.5.3"
@@ -1062,12 +1224,53 @@ dependencies = [
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59b23e92ee4318893fa3fe3e6fb365258efbfe6ac6ab30f090cdcbb7aa37efa9"
[[package]]
name = "regex-syntax"
version = "0.6.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f162c6dd7b008981e4d40210aca20b4bd0f9b60ca9271061b07f78537722f2e1"
[[package]]
name = "rstest"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de1bb486a691878cd320c2f0d319ba91eeaa2e894066d8b5f8f117c000e9d962"
dependencies = [
"futures",
"futures-timer",
"rstest_macros",
"rustc_version",
]
[[package]]
name = "rstest_macros"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "290ca1a1c8ca7edb7c3283bd44dc35dd54fdec6253a3912e201ba1072018fca8"
dependencies = [
"cfg-if",
"proc-macro2",
"quote",
"rustc_version",
"syn 1.0.109",
"unicode-ident",
]
[[package]]
name = "rustc_version"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
dependencies = [
"semver",
]
[[package]]
name = "rusticata-macros"
version = "4.1.0"
@@ -1184,6 +1387,12 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "semver"
version = "1.0.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f79dfe2d285b0488816f30e700a7438c5a73d816b5b7d3ac72fbc48b0d185e03"
[[package]]
name = "serde"
version = "1.0.218"
@@ -1248,12 +1457,27 @@ dependencies = [
"lazy_static",
]
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "siphasher"
version = "0.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38b58827f4464d87d377d175e90bf58eb00fd8716ff0a62f80356b5e61555d0d"
[[package]]
name = "slab"
version = "0.4.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f92a496fb766b417c996b9c5e57daf2f7ad3b0bebe1ccfca4856390e3d3bb67"
dependencies = [
"autocfg",
]
[[package]]
name = "smallvec"
version = "1.14.0"
@@ -1302,6 +1526,7 @@ dependencies = [
"flate2",
"hex",
"hkdf",
"htp",
"ipsec-parser",
"kerberos-parser",
"lazy_static",
@@ -1510,9 +1735,9 @@ dependencies = [
[[package]]
name = "time"
version = "0.3.37"
version = "0.3.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35e7868883861bd0e56d9ac6efcaaca0d6d5d82a2a7ec8209ff492c07cf37b21"
checksum = "5dfd88e563464686c916c7e46e623e520ddc6d79fa6641390f2e3fa86e83e885"
dependencies = [
"deranged",
"itoa",
@@ -1531,9 +1756,9 @@ checksum = "ef927ca75afb808a4d64dd374f00a2adf8d0fcff8e7b184af886c3c87ec4a3f3"
[[package]]
name = "time-macros"
version = "0.2.19"
version = "0.2.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2834e6017e3e5e4b9834939793b282bc03b37a3336245fa820e35e233e2a85de"
checksum = "3f252a68540fde3a3877aeea552b832b40ab9a69e318efd078774a01ddee1ccf"
dependencies = [
"num-conv",
"time-core",
@@ -1677,6 +1902,12 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "vcpkg"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
[[package]]
name = "version_check"
version = "0.9.5"
rust/Cargo.toml.in
+3
View File
@@ -13,6 +13,7 @@ members = [
"suricatactl",
"suricatasc",
"sys",
"htp",
]
default-members = [
@@ -84,5 +85,7 @@ suricata-sys = { path = "./sys", version = "@PACKAGE_VERSION@" }
suricata-lua-sys = { version = "0.1.0-alpha.6" }
htp = { path = "./htp", version = "2.0.0" }
[dev-dependencies]
test-case = "~3.3.1"
rust/Makefile.am
+23 -2
View File
@@ -2,17 +2,19 @@ SUBDIRS = sys \
suricatasc \
suricatactl
EXTRA_DIST = src derive \
EXTRA_DIST = src derive htp \
.cargo/config.toml.in \
cbindgen.toml \
dist/rust-bindings.h \
dist/htp/htp_rs.h \
vendor \
Cargo.toml Cargo.lock \
derive/Cargo.toml \
sys \
sys/Cargo.toml \
suricatasc \
suricatactl
suricatactl \
htp/Cargo.toml
if !DEBUG
RELEASE = --release
@@ -71,6 +73,7 @@ all-local: Cargo.toml
$(RUST_SURICATA_LIBDIR)/${RUST_SURICATA_LIBNAME}; \
fi
$(MAKE) gen/rust-bindings.h
$(MAKE) gen/htp/htp_rs.h
install-exec-local:
install -d -m 0755 "$(DESTDIR)$(bindir)"
@@ -152,6 +155,15 @@ else
gen/rust-bindings.h:
endif
if HAVE_CBINDGEN
gen/htp/htp_rs.h: $(RUST_SURICATA_LIB) htp/cbindgen.toml
cd $(abs_top_srcdir)/rust/htp && \
cbindgen --config $(abs_top_srcdir)/rust/htp/cbindgen.toml \
--quiet --verify --output $(abs_top_builddir)/rust/gen/htp/htp_rs.h || true
else
gen/htp/htp_rs.h:
endif
doc:
CARGO_HOME=$(CARGO_HOME) $(CARGO) doc --all-features --no-deps
@@ -163,6 +175,15 @@ else
dist/rust-bindings.h:
endif
if HAVE_CBINDGEN
dist/htp/htp_rs.h:
cd $(abs_top_srcdir)/rust/htp && \
cbindgen --config cbindgen.toml \
--quiet --output $(abs_top_builddir)/rust/dist/htp/htp_rs.h
else
dist/htp/htp_rs.h:
endif
Cargo.toml: Cargo.toml.in
update-lock: Cargo.toml
rust/htp/.gitignore
+1
View File
@@ -0,0 +1 @@
!Cargo.toml
rust/htp/Cargo.toml
+40
View File
@@ -0,0 +1,40 @@
[package]
name = "htp"
authors = ["ivanr = Ivan Ristic <[email protected]>", "cccs = Canadian Centre for Cyber Security"]
version = "2.0.0"
publish = false
edition = "2021"
autobins = false
license-file = "LICENSE"
description = "Security Aware HTP Protocol parsing library"
readme = "README.md"
repository = "https://github.com/CybercentreCanada/libhtp-rs-internal"
homepage = "https://github.com/CybercentreCanada/libhtp-rs-internal"
keywords = ["parser", "HTTP", "protocol", "network", "api"]
categories = ["parsing", "network-programming"]
include = [
"Cargo.toml",
"LICENSE",
"README.md",
"src/**/*.rs",
"cbindgen.toml",
]
[lib]
crate-type = ["staticlib", "rlib", "cdylib"]
[dependencies]
base64 = "0.22.1"
bstr = "1.6.0"
libc = "0.2"
nom = "7.1.1"
lzma-rs = { version = "0.2.0", features = ["stream"] }
flate2 = { version = "~1.0.35", features = ["zlib-default"], default-features = false }
lazy_static = "1.4.0"
time = "=0.3.36"
[dev-dependencies]
rstest = "0.17.0"
[build-dependencies]
cdylib-link-lines = "0.1.5"
rust/htp/LICENSE
+31
View File
@@ -0,0 +1,31 @@
Copyright (c) 2009-2010 Open Information Security Foundation
Copyright (c) 2010-2013 Qualys, Inc.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
- Neither the name of the Qualys, Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
rust/htp/README.md
+46
View File
@@ -0,0 +1,46 @@
# LibHTP
---
Copyright 2009-2010 Open Information Security Foundation
Copyright 2010-2013 Qualys, Inc.
---
LibHTP is a security-aware parser for the HTTP protocol and the related bits
and pieces. The goal of the project is mainly to support the Suricata use case.
Other use cases might not be fully supported, and we encourage you to cover these.
See the LICENSE file distributed with this work for information
regarding licensing, copying and copyright ownership.
# Usage
Start using libHTP by including it in your project's `Cargo.toml`
dependencies. The base library will also be required for using common
types.
**The minimum supported version of `rustc` is `1.58.1`.**
## Example
```
[dependencies]
htp = "2.0.0"
```
## FFI Support
LibHTP has a foreign function interface for use in C/C++ projects.
FFI Support can be enabled by building with the `cbindgen` feature.
```
# Install cbindgen which is required to generate headers
cargo install --force cbindgen
# Build headers and shared objects
make
```
## LICENSE
LibHTP is licensed under the BSD 3-Clause license (also known as "BSD New" and
"BSD Simplified".) The complete text of the license is enclosed in the file LICENSE.
rust/htp/cbindgen.toml
+64
View File
@@ -0,0 +1,64 @@
language = "C"
# Header wrapping options
#header = "LICENSE here"
#trailer = ""
include_guard = "_HTP_H"
autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Do NOT modify manually */"
#include_version = true
#sys_includes = [] # Sys headers
includes = []
no_includes = false
cpp_compat = true
#after_includes = ""
# Code style
#braces = "SameLine"
#line_length = 100
#tab_wideth = 2
#documentation_style = auto
# Codegen
style = "both"
after_includes = """
#define htp_url_encoding_handling_t HtpUrlEncodingHandling
#define htp_log_code_t HtpLogCode
"""
[export.rename]
"HtpFlags" = "HTP_FLAGS"
"Config" = "htp_cfg_t"
"Connection" = "htp_conn_t"
"ConnectionParser" = "htp_connp_t"
"Header" = "htp_header_t"
"Headers" = "htp_headers_t"
"Param" = "htp_param_t"
"Data" = "htp_tx_data_t"
"Transaction" = "htp_tx_t"
"Transactions" = "htp_txs_t"
"Uri" = "htp_uri_t"
"Bstr" = "bstr"
"Table" = "htp_table_t"
"Log" = "htp_log_t"
"timeval" = "struct timeval"
"Logs" = "htp_logs_t"
[export]
include = [
"HtpUrlEncodingHandling",
"HtpLogCode",
"HtpFlags",
]
[enum]
rename_variants = "QualifiedScreamingSnakeCase"
prefix_with_name = false
[macro_expansion]
bitflags = true
# Rust parsing options
[parse]
parse_deps = false
clean = false
rust/htp/fuzz/Cargo.toml
+25
View File
@@ -0,0 +1,25 @@
[package]
name = "htp-fuzz"
version = "0.0.1"
authors = ["Automatically generated"]
publish = false
edition = "2018"
[package.metadata]
cargo-fuzz = true
[dependencies.htp]
path = ".."
[dependencies.libfuzzer-sys]
git = "https://github.com/rust-fuzz/libfuzzer-sys.git"
[dependencies]
# Prevent this from interfering with workspaces
[workspace]
members = ["."]
[[bin]]
name = "fuzz_htp_rs"
path = "fuzz_targets/fuzz_htp.rs"
rust/htp/fuzz/fuzz_targets/fuzz_htp.rs
+14
View File
@@ -0,0 +1,14 @@
#![allow(non_snake_case)]
#![no_main]
#[macro_use] extern crate libfuzzer_sys;
extern crate htp;
use htp::test::{Test, TestConfig};
use std::env;
fuzz_target!(|data: &[u8]| {
let mut t = Test::new(TestConfig());
t.run_slice(data);
});
rust/htp/src/bstr.rs
+475
View File
@@ -0,0 +1,475 @@
use bstr::{BString, ByteSlice};
use core::cmp::Ordering;
use std::ops::{Deref, DerefMut};
/// Bstr is a convenience wrapper around binary data that adds string-like functions.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Bstr {
// Wrap a BString under the hood. We want to be able to
// implement behaviours on top of this if needed, so we wrap
// it instead of exposing it directly in our public API.
s: BString,
}
impl Default for Bstr {
fn default() -> Self {
Self {
s: BString::from(Vec::new()),
}
}
}
impl Bstr {
/// Make a new owned Bstr
pub(crate) fn new() -> Self {
Bstr {
s: BString::from(Vec::new()),
}
}
/// Make a new owned Bstr with given capacity
pub(crate) fn with_capacity(len: usize) -> Self {
Bstr {
s: BString::from(Vec::with_capacity(len)),
}
}
/// Compare this bstr with the given slice
pub(crate) fn cmp_slice<B: AsRef<[u8]>>(&self, other: B) -> Ordering {
self.as_slice().cmp(other.as_ref())
}
/// Return true if self is equal to other
pub(crate) fn eq_slice<B: AsRef<[u8]>>(&self, other: B) -> bool {
self.cmp_slice(other) == Ordering::Equal
}
/// Compare bstr with the given slice, ingnoring ascii case.
pub(crate) fn cmp_nocase<B: AsRef<[u8]>>(&self, other: B) -> bool {
let lefts: &[u8] = self.as_ref();
let mut lefts = lefts.iter();
let mut rights = other.as_ref().iter();
loop {
match (lefts.next(), rights.next()) {
(None, None) => {
return true;
}
(Some(l), Some(r)) => {
if !l.eq_ignore_ascii_case(r) {
return false;
}
}
_ => {
return false;
}
}
}
}
/// Case insensitive comparison between self and other, ignoring any zeros in self
pub(crate) fn cmp_nocase_nozero<B: AsRef<[u8]>>(&self, other: B) -> bool {
let lefts: &[u8] = self.as_ref();
let mut lefts = lefts.iter().filter(|c| (**c) > 0);
let mut rights = other.as_ref().iter();
loop {
match (lefts.next(), rights.next()) {
(None, None) => {
return true;
}
(Some(l), Some(r)) => {
if !l.eq_ignore_ascii_case(r) {
return false;
}
}
_ => {
// TODO trim ?
return false;
}
}
}
}
/// Extend this bstr with the given slice
pub(crate) fn add<B: AsRef<[u8]>>(&mut self, other: B) {
self.extend_from_slice(other.as_ref())
}
/// Extend the bstr as much as possible without growing
#[cfg(test)]
pub(crate) fn add_noex<B: AsRef<[u8]>>(&mut self, other: B) {
let len = std::cmp::min(self.capacity() - self.len(), other.as_ref().len());
self.add(&other.as_ref()[..len]);
}
/// Return true if this bstr starts with other
#[cfg(test)]
pub(crate) fn starts_with<B: AsRef<[u8]>>(&self, other: B) -> bool {
self.as_slice().starts_with(other.as_ref())
}
/// Return true if this bstr starts with other, ignoring ascii case
pub(crate) fn starts_with_nocase<B: AsRef<[u8]>>(&self, other: B) -> bool {
if self.len() < other.as_ref().len() {
return false;
}
let len: usize = std::cmp::min(self.len(), other.as_ref().len());
self.as_slice()[..len].eq_ignore_ascii_case(&other.as_ref()[..len])
}
/// Find the index of the given slice
#[cfg(test)]
pub(crate) fn index_of<B: AsRef<[u8]>>(&self, other: B) -> Option<usize> {
self.find(other.as_ref())
}
/// Find the index of the given slice ignoring ascii case
pub(crate) fn index_of_nocase<B: AsRef<[u8]>>(&self, other: B) -> Option<usize> {
let src = self.as_slice();
let mut haystack = LowercaseIterator::new(&src);
let needle = other.as_ref().to_ascii_lowercase();
haystack.index_of(&needle)
}
/// Find the index of the given slice ignoring ascii case and any zeros in self
pub(crate) fn index_of_nocase_nozero<B: AsRef<[u8]>>(&self, other: B) -> Option<usize> {
let src = self.as_slice();
let mut haystack = LowercaseNoZeroIterator::new(&src);
let needle = other.as_ref().to_ascii_lowercase();
haystack.index_of(&needle)
}
}
// Trait Implementations for Bstr
/// Let callers access BString functions
impl Deref for Bstr {
type Target = BString;
fn deref(&self) -> &Self::Target {
&self.s
}
}
/// Let callers access mutable BString functions
impl DerefMut for Bstr {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.s
}
}
impl From<&[u8]> for Bstr {
fn from(src: &[u8]) -> Self {
Bstr {
s: BString::from(src),
}
}
}
impl From<&str> for Bstr {
fn from(src: &str) -> Self {
src.as_bytes().into()
}
}
impl From<Vec<u8>> for Bstr {
fn from(src: Vec<u8>) -> Self {
Bstr {
s: BString::from(src),
}
}
}
/// Compare a Bstr to a &str byte for byte
impl PartialEq<&str> for Bstr {
fn eq(&self, rhs: &&str) -> bool {
self.as_bytes() == rhs.as_bytes()
}
}
/// A trait that lets us find the byte index of slices in a generic way.
///
/// This layer of abstraction is motivated by the need to find needle in
/// haystack when we want to perform case sensitive, case insensitive, and
/// case insensitive + zero skipping. All of these algorithms are identical
/// except we compare the needle bytes with the src bytes in different ways,
/// and in the case of zero skipping we want to pretend that zero bytes in
/// the haystack do not exist. So we define iterators for each of lowercase
/// and lowercase + zero skipping, and then implement this trait for both of
/// those, and then define the search function in terms of this trait.
trait SubIterator: Iterator<Item = u8> {
/// Return a new iterator of the same type starting at the current byte index
fn subiter(&self) -> Self;
/// Return the current byte index into the iterator
fn index(&self) -> usize;
/// Find the given needle in self and return the byte index
fn index_of(&mut self, needle: impl AsRef<[u8]>) -> Option<usize>;
}
/// Find the byte index of the given slice in the source.
///
/// Someday an enterprising soul can implement this function inside SubIterator
/// directly (where it arguably belongs), but this involves handling dyn Self,
/// and implementing it this way lets monomorphization emit concrete
/// implementations for each of the two types we actually have.
fn index_of<T: SubIterator, S: AsRef<[u8]>>(haystack: &mut T, needle: &S) -> Option<usize> {
let first = needle.as_ref().first()?;
while let Some(s) = haystack.next() {
if s == *first {
let mut test = haystack.subiter();
let mut equal = false;
for cmp_byte in needle.as_ref().as_bytes() {
equal = Some(*cmp_byte) == test.next();
if !equal {
break;
}
}
if equal {
return Some(haystack.index());
}
}
}
None
}
/// A convenience iterator for anything that satisfies AsRef<[u8]>
/// that yields lowercase ascii bytes and skips null bytes
struct LowercaseNoZeroIterator<'a, T: AsRef<[u8]>> {
src: &'a T,
idx: usize,
first: bool,
}
impl<'a, T: AsRef<[u8]>> LowercaseNoZeroIterator<'a, T> {
fn new(src: &'a T) -> Self {
LowercaseNoZeroIterator {
src,
idx: 0,
first: true,
}
}
}
impl<T: AsRef<[u8]>> Iterator for LowercaseNoZeroIterator<'_, T> {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
loop {
if self.first {
self.first = false;
} else {
self.idx += 1;
}
let next = self
.src
.as_ref()
.get(self.idx)
.map(|c| c.to_ascii_lowercase());
if next != Some(0) {
break next;
}
}
}
}
impl<T: AsRef<[u8]>> SubIterator for LowercaseNoZeroIterator<'_, T> {
fn subiter(&self) -> Self {
LowercaseNoZeroIterator {
src: self.src,
idx: self.idx,
first: true,
}
}
fn index(&self) -> usize {
self.idx
}
fn index_of(&mut self, needle: impl AsRef<[u8]>) -> Option<usize> {
index_of(self, &needle)
}
}
/// A convenience iterator for anything that satisfies AsRef<[u8]>
/// that yields lowercase ascii bytes
struct LowercaseIterator<'a, T: AsRef<[u8]>> {
src: &'a T,
idx: usize,
first: bool,
}
impl<'a, T: AsRef<[u8]>> LowercaseIterator<'a, T> {
fn new(src: &'a T) -> Self {
LowercaseIterator {
src,
idx: 0,
first: true,
}
}
}
impl<T: AsRef<[u8]>> Iterator for LowercaseIterator<'_, T> {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
if self.first {
self.first = false;
} else {
self.idx += 1;
}
self.src
.as_ref()
.get(self.idx)
.map(|c| c.to_ascii_lowercase())
}
}
impl<T: AsRef<[u8]>> SubIterator for LowercaseIterator<'_, T> {
fn subiter(&self) -> Self {
LowercaseIterator {
src: self.src,
idx: self.idx,
first: true,
}
}
fn index(&self) -> usize {
self.idx
}
fn index_of(&mut self, needle: impl AsRef<[u8]>) -> Option<usize> {
index_of(self, &needle)
}
}
#[cfg(test)]
mod tests {
use crate::bstr::*;
use core::cmp::Ordering;
use rstest::rstest;
#[test]
fn Compare() {
let b = Bstr::from("ABCDefgh");
// direct equality
assert_eq!(Ordering::Equal, b.cmp_slice("ABCDefgh"));
// case sensitive
assert_ne!(Ordering::Equal, b.cmp_slice("abcdefgh"));
// src shorter than dst
assert_eq!(Ordering::Less, b.cmp_slice("ABCDefghi"));
// src longer than dst
assert_eq!(Ordering::Greater, b.cmp_slice("ABCDefg"));
// case less
assert_eq!(Ordering::Less, b.cmp_slice("abcdefgh"));
// case greater
assert_eq!(Ordering::Greater, b.cmp_slice("ABCDEFGH"));
}
#[test]
fn CompareNocase() {
let b = Bstr::from("ABCDefgh");
assert!(b.cmp_nocase("ABCDefgh"));
assert!(b.cmp_nocase("abcdefgh"));
assert!(b.cmp_nocase("ABCDEFGH"));
assert!(!b.cmp_nocase("ABCDefghi"));
assert!(!b.cmp_nocase("ABCDefg"));
}
#[test]
fn CompareNocaseNozero() {
// nocase_nozero only applies to the source string. The caller
// is not expected to pass in a search string with nulls in it.
let b = Bstr::from("A\x00B\x00\x00C\x00Defg\x00h");
assert!(b.cmp_nocase_nozero("ABCDefgh"));
assert!(b.cmp_nocase_nozero("abcdefgh"));
assert!(b.cmp_nocase_nozero("ABCDEFGH"));
assert!(!b.cmp_nocase_nozero("ABCDefghi"));
assert!(!b.cmp_nocase_nozero("ABCDefg"));
}
#[rstest]
#[case("abc", "defgh", "abcdefgh")]
#[case("ABC", "DEFGH", "ABCDEFGH")]
#[case("aBc", "Defgh", "aBcDefgh")]
#[case(
"TestLongerDataBc",
"Defghikjlmnopqrstuvwxyz",
"TestLongerDataBcDefghikjlmnopqrstuvwxyz"
)]
fn test_add(#[case] input: &str, #[case] input_add: &str, #[case] expected: &str) {
let mut b = Bstr::from(input);
b.add(input_add);
assert_eq!(b.cmp_slice(expected), Ordering::Equal);
}
#[rstest]
#[case(10, "abcd", "efghij", "abcdefghij")]
#[case(5, "ABcd", "efgh", "ABcde")]
#[case(4, "AbCd", "EFGH", "AbCd")]
#[case(20, "abcd", "efGHij", "abcdefGHij")]
fn test_add_no_ex(
#[case] capacity: usize, #[case] input: &str, #[case] input_add: &str,
#[case] expected: &str,
) {
let mut b = Bstr::with_capacity(capacity);
b.add_noex(input);
b.add_noex(input_add);
assert_eq!(b.cmp_slice(expected), Ordering::Equal);
}
#[test]
fn StartsWith() {
let b = Bstr::from("ABCD");
assert!(b.starts_with("AB"));
assert!(!b.starts_with("ab"));
assert!(!b.starts_with("Ab"));
assert!(!b.starts_with("aB"));
assert!(!b.starts_with("CD"));
}
#[test]
fn StartsWithNocase() {
let b = Bstr::from("ABCD");
assert!(b.starts_with_nocase("AB"));
assert!(b.starts_with_nocase("ab"));
assert!(b.starts_with_nocase("Ab"));
assert!(b.starts_with_nocase("aB"));
assert!(!b.starts_with_nocase("CD"));
}
#[test]
fn IndexOf() {
let b = Bstr::from("ABCDefgh");
assert_eq!(Some(4), b.index_of("e"));
assert_eq!(Some(0), b.index_of("A"));
assert_eq!(Some(7), b.index_of("h"));
assert_eq!(Some(3), b.index_of("De"));
assert_eq!(None, b.index_of("z"));
assert_eq!(None, b.index_of("a"));
assert_eq!(None, b.index_of("hi"));
}
#[test]
fn IndexOfNocase() {
let b = Bstr::from("ABCDefgh");
assert_eq!(Some(4), b.index_of_nocase("E"));
assert_eq!(Some(0), b.index_of_nocase("a"));
assert_eq!(Some(0), b.index_of_nocase("A"));
assert_eq!(Some(7), b.index_of_nocase("H"));
assert_eq!(Some(3), b.index_of_nocase("dE"));
assert_eq!(None, b.index_of_nocase("z"));
assert_eq!(None, b.index_of_nocase("Hi"));
}
#[test]
fn IndexOfNocaseNozero() {
let b = Bstr::from("A\x00B\x00\x00C\x00Defg\x00h");
assert_eq!(Some(8), b.index_of_nocase_nozero("E"));
assert_eq!(Some(0), b.index_of_nocase_nozero("a"));
assert_eq!(Some(0), b.index_of_nocase_nozero("A"));
assert_eq!(Some(12), b.index_of_nocase_nozero("H"));
assert_eq!(Some(7), b.index_of_nocase_nozero("dE"));
assert_eq!(Some(2), b.index_of_nocase_nozero("bc"));
assert_eq!(None, b.index_of_nocase_nozero("z"));
assert_eq!(None, b.index_of_nocase_nozero("Hi"));
assert_eq!(None, b.index_of_nocase_nozero("ghi"));
}
}
rust/htp/src/c_api/bstr.rs
+196
View File
@@ -0,0 +1,196 @@
use crate::bstr::Bstr;
use core::cmp::Ordering;
use std::{boxed::Box, ffi::CStr};
/// Allocate a zero-length bstring, reserving space for at least size bytes.
fn bstr_alloc(len: libc::size_t) -> *mut Bstr {
let b = Bstr::with_capacity(len);
let boxed = Box::new(b);
Box::into_raw(boxed)
}
/// Deallocate the supplied bstring instance. Allows NULL on input.
/// # Safety
/// This function is unsafe because improper use may lead to memory problems. For example, a double-free may occur if the function is called twice on the same raw pointer.
#[no_mangle]
pub unsafe extern "C" fn bstr_free(b: *mut Bstr) {
if !b.is_null() {
drop(Box::from_raw(b));
}
}
/// Return the length of the string
/// # Safety
/// x must be properly intialized: not NULL, dangling, or misaligned
#[no_mangle]
pub unsafe extern "C" fn bstr_len(x: *const Bstr) -> libc::size_t {
(*x).len()
}
/// Return a pointer to the bstr payload
/// # Safety
/// x must be properly intialized: not NULL, dangling, or misaligned
#[no_mangle]
pub unsafe extern "C" fn bstr_ptr(x: *const Bstr) -> *mut libc::c_uchar {
(*x).as_ptr() as *mut u8
}
/// Return the capacity of the string
/// # Safety
/// x must be properly intialized: not NULL, dangling, or misaligned
#[no_mangle]
pub unsafe extern "C" fn bstr_size(x: *const Bstr) -> libc::size_t {
(*x).capacity()
}
/// Case-sensitive comparison of a bstring and a NUL-terminated string.
/// returns -1 if b is less than c
/// 0 if b is equal to c
/// 1 if b is greater than c
/// # Safety
/// b and c must be properly intialized: not NULL, dangling, or misaligned.
/// c must point to memory that contains a valid nul terminator byte at the end of the string
#[no_mangle]
pub unsafe extern "C" fn bstr_cmp_c(b: *const Bstr, c: *const libc::c_char) -> libc::c_int {
let cs = CStr::from_ptr(c);
match (*b).cmp_slice(cs.to_bytes()) {
Ordering::Less => -1,
Ordering::Equal => 0,
Ordering::Greater => 1,
}
}
/// Case-indensitive comparison of a bstring and a NUL-terminated string.
/// returns -1 if b is less than c
/// 0 if b is equal to c
/// 1 if b is greater than c
/// # Safety
/// b and c must be properly intialized: not NULL, dangling, or misaligned.
/// c must point to memory that contains a valid nul terminator byte at the end of the string
#[no_mangle]
pub unsafe extern "C" fn bstr_cmp_c_nocase(b: *const Bstr, c: *const libc::c_char) -> bool {
let cs = CStr::from_ptr(c);
(*b).cmp_nocase(cs.to_bytes())
}
/// Create a new bstring by copying the provided NUL-terminated string
/// # Safety
/// cstr must be properly intialized: not NULL, dangling, or misaligned.
/// cstr must point to memory that contains a valid nul terminator byte at the end of the string
#[no_mangle]
pub unsafe extern "C" fn bstr_dup_c(cstr: *const libc::c_char) -> *mut Bstr {
let cs = CStr::from_ptr(cstr).to_bytes();
let new = bstr_alloc(cs.len());
(*new).add(cs);
new
}
/// Create a new NUL-terminated string out of the provided bstring. If NUL bytes
/// are contained in the bstring, each will be replaced with "\0" (two characters).
/// The caller is responsible to keep track of the allocated memory area and free
/// it once it is no longer needed.
/// returns The newly created NUL-terminated string, or NULL in case of memory
/// allocation failure.
/// # Safety
/// b must be properly intialized and not dangling nor misaligned.
#[no_mangle]
pub unsafe extern "C" fn bstr_util_strdup_to_c(b: *const Bstr) -> *mut libc::c_char {
if b.is_null() {
return std::ptr::null_mut();
}
let src = std::slice::from_raw_parts(bstr_ptr(b), bstr_len(b));
// Since the memory returned here is just a char* and the caller will
// free() it we have to use malloc() here.
// So we allocate enough space for doubled NULL bytes plus the trailing NULL.
let mut null_count = 1;
for byte in src {
if *byte == 0 {
null_count += 1;
}
}
let newlen = bstr_len(b) + null_count;
let mem = libc::malloc(newlen) as *mut libc::c_char;
if mem.is_null() {
return std::ptr::null_mut();
}
let dst: &mut [libc::c_char] = std::slice::from_raw_parts_mut(mem, newlen);
let mut dst_idx = 0;
for byte in src {
if *byte == 0 {
dst[dst_idx] = '\\' as libc::c_char;
dst_idx += 1;
dst[dst_idx] = '0' as libc::c_char;
} else {
dst[dst_idx] = *byte as libc::c_char;
}
dst_idx += 1;
}
dst[dst_idx] = 0;
mem
}
#[cfg(test)]
mod test {
use super::*;
use std::ffi::CString;
macro_rules! cstr {
( $x:expr ) => {{
CString::new($x).unwrap()
}};
}
#[test]
fn Bstr_Alloc() {
unsafe {
let p1 = bstr_alloc(10);
assert_eq!(10, bstr_size(p1));
assert_eq!(0, bstr_len(p1));
bstr_free(p1);
}
}
#[test]
fn Bstr_DupC() {
unsafe {
let p1 = bstr_dup_c(cstr!("arfarf").as_ptr());
assert_eq!(6, bstr_size(p1));
assert_eq!(6, bstr_len(p1));
assert_eq!(
0,
libc::memcmp(
cstr!("arfarf").as_ptr() as *const core::ffi::c_void,
bstr_ptr(p1) as *const core::ffi::c_void,
6
)
);
bstr_free(p1);
}
}
#[test]
fn Bstr_UtilDupToC() {
unsafe {
let s = Bstr::from(b"ABCDEFGHIJKL\x00NOPQRST" as &[u8]);
let c = bstr_util_strdup_to_c(&s);
let e = CString::new("ABCDEFGHIJKL\\0NOPQRST").unwrap();
assert_eq!(0, libc::strcmp(e.as_ptr(), c));
libc::free(c as *mut core::ffi::c_void);
}
}
#[test]
fn Bstr_CmpC() {
unsafe {
let p1 = Bstr::from("arfarf");
assert_eq!(0, bstr_cmp_c(&p1, cstr!("arfarf").as_ptr()));
assert_eq!(-1, bstr_cmp_c(&p1, cstr!("arfarf2").as_ptr()));
assert_eq!(1, bstr_cmp_c(&p1, cstr!("arf").as_ptr()));
assert_eq!(-1, bstr_cmp_c(&p1, cstr!("not equal").as_ptr()));
}
}
}
rust/htp/src/c_api/config.rs
+498
View File
@@ -0,0 +1,498 @@
#![deny(missing_docs)]
use crate::{
config::{Config, HtpServerPersonality, HtpUrlEncodingHandling},
hook::{DataExternalCallbackFn, TxExternalCallbackFn},
HtpStatus,
};
use std::convert::TryInto;
/// Creates a new configuration structure. Configuration structures created at
/// configuration time must not be changed afterwards in order to support lock-less
/// copying.
#[no_mangle]
pub extern "C" fn htp_config_create() -> *mut Config {
let cfg: Config = Config::default();
let b = Box::new(cfg);
Box::into_raw(b)
}
/// Destroy a configuration structure.
/// # Safety
/// This function is unsafe because improper use may lead to memory problems. For example, a double-free may occur if the function is called twice on the same raw pointer.
#[no_mangle]
pub unsafe extern "C" fn htp_config_destroy(cfg: *mut Config) {
if !cfg.is_null() {
drop(Box::from_raw(cfg));
}
}
/// Registers a REQUEST_BODY_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_body_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_body_data.register_extern(cbk_fn)
}
}
/// Registers a REQUEST_COMPLETE callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_complete(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_complete.register_extern(cbk_fn)
}
}
/// Registers a REQUEST_HEADER_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_header_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_header_data.register_extern(cbk_fn)
}
}
/// Registers a REQUEST_LINE callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_line(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_line.register_extern(cbk_fn)
}
}
/// Registers a REQUEST_START callback, which is invoked every time a new
/// request begins and before any parsing is done.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_start(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_start.register_extern(cbk_fn)
}
}
/// Registers a HTP_REQUEST_TRAILER callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_trailer(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_trailer.register_extern(cbk_fn)
}
}
/// Registers a REQUEST_TRAILER_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_request_trailer_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_request_trailer_data.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_BODY_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_body_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_body_data.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_COMPLETE callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_complete(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_complete.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_HEADER_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_header_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_header_data.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_START callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_start(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_start.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_TRAILER callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_trailer(
cfg: *mut Config, cbk_fn: TxExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_trailer.register_extern(cbk_fn)
}
}
/// Registers a RESPONSE_TRAILER_DATA callback.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_register_response_trailer_data(
cfg: *mut Config, cbk_fn: DataExternalCallbackFn,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.hook_response_trailer_data.register_extern(cbk_fn)
}
}
/// Configures whether backslash characters are treated as path segment separators. They
/// are not on Unix systems, but are on Windows systems. If this setting is enabled, a path
/// such as "/one\two/three" will be converted to "/one/two/three".
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_backslash_convert_slashes(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_backslash_convert_slashes(enabled == 1)
}
}
/// Sets the replacement character that will be used to in the lossy best-fit
/// mapping from multi-byte to single-byte streams. The question mark character
/// is used as the default replacement byte.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_bestfit_replacement_byte(cfg: *mut Config, b: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_bestfit_replacement_byte(b as u8)
}
}
/// Configures the maximum compression bomb size LibHTP will decompress.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_compression_bomb_limit(
cfg: *mut Config, bomblimit: libc::size_t,
) {
if let Ok(bomblimit) = bomblimit.try_into() {
if let Some(cfg) = cfg.as_mut() {
cfg.compression_options.set_bomb_limit(bomblimit)
}
}
}
/// Configures the maximum compression time LibHTP will allow.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_compression_time_limit(
cfg: *mut Config, timelimit: libc::c_uint,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.compression_options.set_time_limit(timelimit)
}
}
/// Configures whether input data will be converted to lowercase. Useful for handling servers with
/// case-insensitive filesystems.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_convert_lowercase(cfg: *mut Config, enabled: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_convert_lowercase(enabled == 1)
}
}
/// Configures the maximum size of the buffer LibHTP will use when all data is not available
/// in the current buffer (e.g., a very long header line that might span several packets). This
/// limit is controlled by the field_limit parameter.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_field_limit(cfg: *mut Config, field_limit: libc::size_t) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_field_limit(field_limit)
}
}
/// Configures the maximum memlimit LibHTP will pass to liblzma.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_lzma_memlimit(cfg: *mut Config, memlimit: libc::size_t) {
if let Some(cfg) = cfg.as_mut() {
cfg.compression_options.set_lzma_memlimit(memlimit)
}
}
/// Configures the maximum number of lzma layers to pass to the decompressor.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_lzma_layers(cfg: *mut Config, limit: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.compression_options.set_lzma_layers(if limit <= 0 {
None
} else {
limit.try_into().ok()
})
}
}
/// Configures the maximum number of live transactions per connection
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_max_tx(cfg: *mut Config, limit: u32) {
if let Some(cfg) = cfg.as_mut() {
cfg.max_tx = limit;
}
}
/// Configures the maximum number of headers in one transaction
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_number_headers_limit(cfg: *mut Config, limit: u32) {
if let Some(cfg) = cfg.as_mut() {
cfg.number_headers_limit = limit;
}
}
/// Configures how the server reacts to encoded NUL bytes. Some servers will stop at
/// at NUL, while some will respond with 400 or 404. When the termination option is not
/// used, the NUL byte will remain in the path.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_nul_encoded_terminates(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_nul_encoded_terminates(enabled == 1)
}
}
/// Configures the handling of raw NUL bytes. If enabled, raw NUL terminates strings.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_nul_raw_terminates(cfg: *mut Config, enabled: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_nul_raw_terminates(enabled == 1)
}
}
/// Enable or disable request cookie parsing. Enabled by default.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_parse_request_cookies(
_cfg: *mut Config, _parse_request_cookies: libc::c_int,
) {
// do nothing, but keep API
}
/// Configures whether consecutive path segment separators will be compressed. When enabled, a path
/// such as "/one//two" will be normalized to "/one/two". Backslash conversion and path segment separator
/// decoding are carried out before compression. For example, the path "/one\\/two\/%5cthree/%2f//four"
/// will be converted to "/one/two/three/four" (assuming all 3 options are enabled).
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_path_separators_compress(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_path_separators_compress(enabled == 1)
}
}
/// Configures whether plus characters are converted to spaces when decoding URL-encoded strings. This
/// is appropriate to do for parameters, but not for URLs. Only applies to contexts where decoding
/// is taking place.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_plusspace_decode(cfg: *mut Config, enabled: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_plusspace_decode(enabled == 1)
}
}
/// Configures whether encoded path segment separators will be decoded. Apache does not do
/// this by default, but IIS does. If enabled, a path such as "/one%2ftwo" will be normalized
/// to "/one/two". If the backslash_separators option is also enabled, encoded backslash
/// characters will be converted too (and subsequently normalized to forward slashes).
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_path_separators_decode(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_path_separators_decode(enabled == 1)
}
}
/// Configures whether request data is decompressed
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_request_decompression(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_request_decompression(enabled == 1)
}
}
/// Configures many layers of compression we try to decompress.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_decompression_layer_limit(
cfg: *mut Config, limit: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_decompression_layer_limit(if limit <= 0 {
None
} else {
limit.try_into().ok()
})
}
}
/// Enable or disable allowing spaces in URIs. Disabled by default.
/// # Safety
/// When calling this method the given cfg must be initialized or NULL.
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_allow_space_uri(cfg: *mut Config, allow_space: bool) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_allow_space_uri(allow_space)
}
}
/// Configure desired server personality.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_server_personality(
cfg: *mut Config, personality: HtpServerPersonality,
) -> HtpStatus {
cfg.as_mut()
.map(|cfg| cfg.set_server_personality(personality).into())
.unwrap_or(HtpStatus::ERROR)
}
/// Configures whether %u-encoded sequences are decoded. Such sequences
/// will be treated as invalid URL encoding if decoding is not desirable.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_u_encoding_decode(cfg: *mut Config, enabled: libc::c_int) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_u_encoding_decode(enabled == 1)
}
}
/// Configures how the server handles to invalid URL encoding.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_url_encoding_invalid_handling(
cfg: *mut Config, handling: HtpUrlEncodingHandling,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_url_encoding_invalid_handling(handling)
}
}
/// Controls whether the data should be treated as UTF-8 and converted to a single-byte
/// stream using best-fit mapping.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_utf8_convert_bestfit(
cfg: *mut Config, enabled: libc::c_int,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_utf8_convert_bestfit(enabled == 1)
}
}
/// Configures whether to attempt to decode a double encoded query in the normalized uri
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_double_decode_normalized_query(
cfg: *mut Config, set: bool,
) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_double_decode_normalized_query(set)
}
}
/// Configures whether to attempt to decode a double encoded path in the normalized uri
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_double_decode_normalized_path(cfg: *mut Config, set: bool) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_double_decode_normalized_path(set)
}
}
/// Configures whether to normalize URIs into a complete or partial form.
/// Pass `true` to use complete normalized URI or `false` to use partials.
/// # Safety
/// When calling this method, you have to ensure that cfg is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_config_set_normalized_uri_include_all(cfg: *mut Config, set: bool) {
if let Some(cfg) = cfg.as_mut() {
cfg.set_normalized_uri_include_all(set)
}
}
rust/htp/src/c_api/connection.rs
+36
View File
@@ -0,0 +1,36 @@
#![deny(missing_docs)]
use crate::{connection::Connection, log::Log};
/// Returns the request_data_counter
/// # Safety
/// When calling this method, you have to ensure that conn is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_conn_request_data_counter(conn: *const Connection) -> u64 {
conn.as_ref()
.map(|conn| conn.request_data_counter)
.unwrap_or(0)
}
/// Returns the response_data_counter
/// # Safety
/// When calling this method, you have to ensure that conn is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_conn_response_data_counter(conn: *const Connection) -> u64 {
conn.as_ref()
.map(|conn| conn.response_data_counter)
.unwrap_or(0)
}
/// Get the next logged message from the connection
///
/// Returns the next log or NULL on error.
/// The caller must free this result with htp_log_free
/// # Safety
/// When calling this method, you have to ensure that conn is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_conn_next_log(conn: *const Connection) -> *mut Log {
conn.as_ref()
.and_then(|conn| conn.get_next_log())
.map(|log| Box::into_raw(Box::new(log)))
.unwrap_or(std::ptr::null_mut())
}
rust/htp/src/c_api/connection_parser.rs
+308
View File
@@ -0,0 +1,308 @@
#![deny(missing_docs)]
use crate::{
config::Config,
connection::Connection,
connection_parser::{ConnectionParser, HtpStreamState, ParserData},
transaction::Transaction,
};
use std::{
convert::{TryFrom, TryInto},
ffi::CStr,
};
use time::{Duration, OffsetDateTime};
/// Take seconds and microseconds and return a OffsetDateTime
fn datetime_from_sec_usec(sec: i64, usec: i64) -> Option<OffsetDateTime> {
match OffsetDateTime::from_unix_timestamp(sec) {
Ok(date) => Some(date + Duration::microseconds(usec)),
Err(_) => None,
}
}
/// Closes the connection associated with the supplied parser.
///
/// timestamp is optional
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
#[allow(clippy::useless_conversion)]
pub unsafe extern "C" fn htp_connp_close(
connp: *mut ConnectionParser, timestamp: *const libc::timeval,
) {
if let Some(connp) = connp.as_mut() {
connp.close(
timestamp
.as_ref()
.map(|val| datetime_from_sec_usec(val.tv_sec.into(), val.tv_usec.into()))
.unwrap_or(None),
)
}
}
/// Creates a new connection parser using the provided configuration or a default configuration if NULL provided.
/// Note the provided config will be copied into the created connection parser. Therefore, subsequent modification
/// to the original config will have no effect.
///
/// Returns a new connection parser instance, or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_create(cfg: *const Config) -> *mut ConnectionParser {
Box::into_raw(Box::new(ConnectionParser::new(cfg.as_ref().unwrap())))
}
/// Destroys the connection parser, its data structures, as well
/// as the connection and its transactions.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_destroy_all(connp: *mut ConnectionParser) {
drop(Box::from_raw(connp));
}
/// Returns the connection associated with the connection parser.
///
/// Returns Connection instance, or NULL if one is not available.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_connection(connp: *const ConnectionParser) -> *const Connection {
connp
.as_ref()
.map(|val| &val.conn as *const Connection)
.unwrap_or(std::ptr::null())
}
/// Retrieve the user data associated with this connection parser.
/// Returns user data, or NULL if there isn't any.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_user_data(connp: *const ConnectionParser) -> *mut libc::c_void {
connp
.as_ref()
.and_then(|val| val.user_data::<*mut libc::c_void>())
.copied()
.unwrap_or(std::ptr::null_mut())
}
/// Associate user data with the supplied parser.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_set_user_data(
connp: *mut ConnectionParser, user_data: *mut libc::c_void,
) {
if let Some(connp) = connp.as_mut() {
connp.set_user_data(Box::new(user_data))
}
}
/// Opens connection.
///
/// timestamp is optional
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
#[allow(clippy::useless_conversion)]
pub unsafe extern "C" fn htp_connp_open(
connp: *mut ConnectionParser, client_addr: *const libc::c_char, client_port: libc::c_int,
server_addr: *const libc::c_char, server_port: libc::c_int, timestamp: *const libc::timeval,
) {
if let Some(connp) = connp.as_mut() {
connp.open(
client_addr.as_ref().and_then(|client_addr| {
CStr::from_ptr(client_addr)
.to_str()
.ok()
.and_then(|val| val.parse().ok())
}),
client_port.try_into().ok(),
server_addr.as_ref().and_then(|server_addr| {
CStr::from_ptr(server_addr)
.to_str()
.ok()
.and_then(|val| val.parse().ok())
}),
server_port.try_into().ok(),
timestamp
.as_ref()
.map(|val| datetime_from_sec_usec(val.tv_sec.into(), val.tv_usec.into()))
.unwrap_or(None),
)
}
}
/// Closes the connection associated with the supplied parser.
///
/// timestamp is optional
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
#[allow(clippy::useless_conversion)]
pub unsafe extern "C" fn htp_connp_request_close(
connp: *mut ConnectionParser, timestamp: *const libc::timeval,
) {
if let Some(connp) = connp.as_mut() {
connp.request_close(
timestamp
.as_ref()
.map(|val| datetime_from_sec_usec(val.tv_sec.into(), val.tv_usec.into()))
.unwrap_or(None),
)
}
}
/// Process a chunk of inbound client request data
///
/// timestamp is optional
/// Returns HTP_STREAM_STATE_DATA, HTP_STREAM_STATE_ERROR or HTP_STREAM_STATE_DATA_OTHER (see QUICK_START).
/// HTP_STREAM_STATE_CLOSED and HTP_STREAM_STATE_TUNNEL are also possible.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
#[allow(clippy::useless_conversion)]
pub unsafe extern "C" fn htp_connp_request_data(
connp: *mut ConnectionParser, timestamp: *const libc::timeval, data: *const libc::c_void,
len: libc::size_t,
) -> HtpStreamState {
connp
.as_mut()
.map(|connp| {
connp.request_data(
ParserData::from((data as *const u8, len)),
timestamp
.as_ref()
.map(|val| datetime_from_sec_usec(val.tv_sec.into(), val.tv_usec.into()))
.unwrap_or(None),
)
})
.unwrap_or(HtpStreamState::ERROR)
}
/// Process a chunk of outbound (server or response) data.
///
/// timestamp is optional.
/// Returns HTP_STREAM_STATE_OK on state change, HTP_STREAM_STATE_ERROR on error, or HTP_STREAM_STATE_DATA when more data is needed
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
#[allow(clippy::useless_conversion)]
pub unsafe extern "C" fn htp_connp_response_data(
connp: *mut ConnectionParser, timestamp: *const libc::timeval, data: *const libc::c_void,
len: libc::size_t,
) -> HtpStreamState {
connp
.as_mut()
.map(|connp| {
connp.response_data(
ParserData::from((data as *const u8, len)),
timestamp
.as_ref()
.map(|val| datetime_from_sec_usec(val.tv_sec.into(), val.tv_usec.into()))
.unwrap_or(None),
)
})
.unwrap_or(HtpStreamState::ERROR)
}
/// Get the number of transactions processed on this connection.
///
/// Returns the number of transactions or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_tx_size(connp: *const ConnectionParser) -> isize {
connp
.as_ref()
.map(|connp| isize::try_from(connp.tx_size()).unwrap_or(-1))
.unwrap_or(-1)
}
/// Get a transaction.
///
/// Returns the transaction or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_tx(
connp: *mut ConnectionParser, tx_id: usize,
) -> *const Transaction {
connp
.as_ref()
.map(|connp| {
connp
.tx(tx_id)
.map(|tx| {
if tx.is_started() {
tx as *const Transaction
} else {
std::ptr::null()
}
})
.unwrap_or(std::ptr::null())
})
.unwrap_or(std::ptr::null())
}
/// Retrieves the pointer to the active response transaction. In connection
/// parsing mode there can be many open transactions, and up to 2 active
/// transactions at any one time. This is due to HTTP pipelining. Can be NULL.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_get_response_tx(
connp: *mut ConnectionParser,
) -> *const Transaction {
if let Some(connp) = connp.as_mut() {
if let Some(req) = connp.response() {
return req;
}
}
std::ptr::null()
}
/// Retrieves the pointer to the active request transaction. In connection
/// parsing mode there can be many open transactions, and up to 2 active
/// transactions at any one time. This is due to HTTP pipelining. Call be NULL.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_get_request_tx(
connp: *mut ConnectionParser,
) -> *const Transaction {
if let Some(connp) = connp.as_mut() {
if let Some(req) = connp.request() {
return req;
}
}
std::ptr::null()
}
/// Returns the number of bytes consumed from the current data chunks so far or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_request_data_consumed(connp: *const ConnectionParser) -> i64 {
connp
.as_ref()
.map(|connp| connp.request_data_consumed().try_into().ok().unwrap_or(-1))
.unwrap_or(-1)
}
/// Returns the number of bytes consumed from the most recent outbound data chunk. Normally, an invocation
/// of htp_connp_response_data() will consume all data from the supplied buffer, but there are circumstances
/// where only partial consumption is possible. In such cases HTP_STREAM_DATA_OTHER will be returned.
/// Consumed bytes are no longer necessary, but the remainder of the buffer will be need to be saved
/// for later.
/// Returns the number of bytes consumed from the last data chunk sent for outbound processing
/// or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that connp is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_connp_response_data_consumed(connp: *const ConnectionParser) -> i64 {
connp
.as_ref()
.map(|connp| connp.response_data_consumed().try_into().ok().unwrap_or(-1))
.unwrap_or(-1)
}
rust/htp/src/c_api/header.rs
+158
View File
@@ -0,0 +1,158 @@
#![deny(missing_docs)]
use crate::{
bstr::Bstr,
c_api::bstr::bstr_ptr,
transaction::{Header, Headers},
};
use std::convert::TryFrom;
/// Get the first header value matching the key.
///
/// headers: Header table.
/// ckey: Header name to match.
///
/// Returns the header or NULL when not found or on error
/// # Safety
/// When calling this method, you have to ensure that headers is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_headers_get(
headers: *const Headers, ckey: *const libc::c_char,
) -> *const Header {
if let (Some(headers), Some(ckey)) = (headers.as_ref(), ckey.as_ref()) {
headers
.get_nocase_nozero(std::ffi::CStr::from_ptr(ckey).to_bytes())
.map(|value| value as *const Header)
.unwrap_or(std::ptr::null())
} else {
std::ptr::null()
}
}
/// Get the header at a given index.
///
/// headers: Header table.
/// index: Index into the table.
///
/// Returns the header or NULL when not found or on error
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_headers_get_index(
headers: *const Headers, index: usize,
) -> *const Header {
headers
.as_ref()
.map(|headers| {
headers
.elements
.get(index)
.map(|value| value as *const Header)
.unwrap_or(std::ptr::null())
})
.unwrap_or(std::ptr::null())
}
/// Get the size of the headers table.
///
/// headers: Headers table.
///
/// Returns the size or -1 on error
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_headers_size(headers: *const Headers) -> isize {
headers
.as_ref()
.map(|headers| isize::try_from(headers.size()).unwrap_or(-1))
.unwrap_or(-1)
}
/// Get the name of a header.
///
/// header: Header pointer.
///
/// Returns the name or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_name(header: *const Header) -> *const Bstr {
header
.as_ref()
.map(|header| &header.name as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the name of a header as a ptr.
///
/// header: Header pointer.
///
/// Returns the pointer or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_name_ptr(header: *const Header) -> *const u8 {
header
.as_ref()
.map(|header| bstr_ptr(&header.name) as *const u8)
.unwrap_or(std::ptr::null())
}
/// Get the length of a header name.
///
/// tx: Header pointer.
///
/// Returns the length or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_name_len(header: *const Header) -> isize {
header
.as_ref()
.map(|header| isize::try_from(header.name.len()).unwrap_or(-1))
.unwrap_or(-1)
}
/// Get the value of a header.
///
/// tx: Header pointer.
///
/// Returns the value or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_value(header: *const Header) -> *const Bstr {
header
.as_ref()
.map(|header| &header.value as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the value of a header as a ptr.
///
/// tx: Header pointer.
///
/// Returns the pointer or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_value_ptr(header: *const Header) -> *const u8 {
header
.as_ref()
.map(|header| bstr_ptr(&header.value) as *const u8)
.unwrap_or(std::ptr::null())
}
/// Get the length of a header value.
///
/// tx: Header pointer.
///
/// Returns the length or 0 on a NULL pointer.
/// # Safety
/// When calling this method, you have to ensure that header is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_header_value_len(header: *const Header) -> usize {
header
.as_ref()
.map(|header| header.value.len())
.unwrap_or(0)
}
rust/htp/src/c_api/log.rs
+39
View File
@@ -0,0 +1,39 @@
#![deny(missing_docs)]
use crate::log::{HtpLogCode, Log};
use std::{ffi::CString, os::raw::c_char};
/// Get the log's message string
///
/// Returns the log message as a cstring or NULL on error
/// The caller must free this result with htp_free_cstring
/// # Safety
/// When calling this method, you have to ensure that log is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_log_message(log: *const Log) -> *mut c_char {
log.as_ref()
.and_then(|log| CString::new(log.msg.msg.clone()).ok())
.map(|msg| msg.into_raw())
.unwrap_or(std::ptr::null_mut())
}
/// Get a log's message code
///
/// Returns a code or HTP_LOG_CODE_ERROR on error
/// # Safety
/// When calling this method, you have to ensure that log is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_log_code(log: *const Log) -> HtpLogCode {
log.as_ref()
.map(|log| log.msg.code)
.unwrap_or(HtpLogCode::ERROR)
}
/// Free log
/// # Safety
/// This function is unsafe because improper use may lead to memory problems. For example, a double-free may occur if the function is called twice on the same raw pointer.
#[no_mangle]
pub unsafe extern "C" fn htp_log_free(log: *mut Log) {
if !log.is_null() {
drop(Box::from_raw(log));
}
}
rust/htp/src/c_api/mod.rs
+35
View File
@@ -0,0 +1,35 @@
#![deny(missing_docs)]
use crate::util::get_version;
use std::ffi::CString;
/// Functions for working with Bstr.
pub mod bstr;
/// Functions for working with config.
pub mod config;
/// Functions for working with connection.
pub mod connection;
/// Functions for working with connection parser.
pub mod connection_parser;
/// Functions for working with headers.
pub mod header;
/// Functions for working with logs.
pub mod log;
/// Functions for working with transactions.
pub mod transaction;
/// Functions for working with request uri.
pub mod uri;
/// Returns the LibHTP version string.
#[no_mangle]
pub extern "C" fn htp_get_version() -> *const libc::c_char {
get_version().as_ptr() as *const libc::c_char
}
/// Free rust allocated cstring
///
/// # Safety
/// This should only ever be called with a pointer that was earlier obtained by calling [CString::into_raw].
#[no_mangle]
pub unsafe extern "C" fn htp_free_cstring(input: *mut libc::c_char) {
input.as_mut().map(|input| CString::from_raw(input));
}
rust/htp/src/c_api/transaction.rs
+522
View File
@@ -0,0 +1,522 @@
use crate::{
bstr::Bstr, c_api::header::htp_headers_get, connection_parser::ConnectionParser,
request::HtpMethod, transaction::*, uri::Uri,
};
use std::convert::{TryFrom, TryInto};
/// Destroys the supplied transaction.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_destroy(connp: *mut ConnectionParser, tx: *const Transaction) {
if let (Some(connp), Some(tx)) = (connp.as_mut(), tx.as_ref()) {
connp.remove_tx(tx.index)
}
}
/// Get a transaction's normalized parsed uri.
///
/// tx: Transaction pointer.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_normalized_uri(tx: *const Transaction) -> *const Bstr {
if (*tx).cfg.decoder_cfg.normalized_uri_include_all {
tx.as_ref()
.and_then(|tx| tx.complete_normalized_uri.as_ref())
.map(|uri| uri as *const Bstr)
.unwrap_or(std::ptr::null())
} else {
tx.as_ref()
.and_then(|tx| tx.partial_normalized_uri.as_ref())
.map(|uri| uri as *const Bstr)
.unwrap_or(std::ptr::null())
}
}
/// Returns the user data associated with this transaction or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_get_user_data(tx: *const Transaction) -> *mut libc::c_void {
tx.as_ref()
.and_then(|val| val.user_data::<*mut libc::c_void>())
.copied()
.unwrap_or(std::ptr::null_mut())
}
/// Associates user data with this transaction.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_set_user_data(tx: *mut Transaction, user_data: *mut libc::c_void) {
if let Some(tx) = tx.as_mut() {
tx.set_user_data(Box::new(user_data))
}
}
/// Get a transaction's request line.
///
/// tx: Transaction pointer.
///
/// Returns the request line or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_line(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.request_line.as_ref())
.map(|line| line as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's request method.
///
/// tx: Transaction pointer.
///
/// Returns the request method or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_method(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.request_method.as_ref())
.map(|method| method as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the transaction's request method number.
///
/// tx: Transaction pointer.
///
/// Returns the request method number or ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_method_number(tx: *const Transaction) -> HtpMethod {
tx.as_ref()
.map(|tx| tx.request_method_number)
.unwrap_or(HtpMethod::ERROR)
}
/// Get a transaction's request uri.
///
/// tx: Transaction pointer.
///
/// Returns the request uri or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_uri(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.request_uri.as_ref())
.map(|uri| uri as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's request protocol.
///
/// tx: Transaction pointer.
///
/// Returns the protocol or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_protocol(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.request_protocol.as_ref())
.map(|protocol| protocol as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's request protocol number.
///
/// tx: Transaction pointer.
///
/// Returns the protocol number or ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_protocol_number(tx: *const Transaction) -> HtpProtocol {
tx.as_ref()
.map(|tx| tx.request_protocol_number)
.unwrap_or(HtpProtocol::Error)
}
/// Get whether a transaction's protocol is version 0.9.
///
/// tx: Transaction pointer.
///
/// Returns 1 if the version is 0.9 or 0 otherwise. A NULL argument will
/// also result in a return value of 0.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_is_protocol_0_9(tx: *const Transaction) -> i32 {
tx.as_ref().map(|tx| tx.is_protocol_0_9 as i32).unwrap_or(0)
}
/// Get a transaction's parsed uri.
///
/// tx: Transaction pointer.
///
/// Returns the parsed uri or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_parsed_uri(tx: *const Transaction) -> *const Uri {
tx.as_ref()
.and_then(|tx| tx.parsed_uri.as_ref())
.map(|uri| uri as *const Uri)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's request headers.
///
/// tx: Transaction pointer.
///
/// Returns the request headers or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_headers(tx: *const Transaction) -> *const Headers {
tx.as_ref()
.map(|tx| &tx.request_headers as *const Headers)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's request headers size.
///
/// tx: Transaction pointer.
///
/// Returns the size or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_headers_size(tx: *const Transaction) -> isize {
tx.as_ref()
.map(|tx| isize::try_from(tx.request_headers.size()).unwrap_or(-1))
.unwrap_or(-1)
}
/// Get the first request header value matching the key from a transaction.
///
/// tx: Transaction pointer.
/// ckey: Header name to match.
///
/// Returns the header or NULL when not found or on error
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_header(
tx: *const Transaction, ckey: *const libc::c_char,
) -> *const Header {
tx.as_ref()
.map(|tx| htp_headers_get(&tx.request_headers, ckey))
.unwrap_or(std::ptr::null())
}
/// Get the request header at the given index.
///
/// tx: Transaction pointer.
/// index: request header table index.
///
/// Returns the header or NULL on error
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_header_index(
tx: *const Transaction, index: usize,
) -> *const Header {
tx.as_ref()
.map(|tx| {
tx.request_headers
.elements
.get(index)
.map(|value| value as *const Header)
.unwrap_or(std::ptr::null())
})
.unwrap_or(std::ptr::null())
}
/// Get the transaction's request authentication type.
///
/// tx: Transaction pointer.
///
/// Returns the auth type or HTP_AUTH_ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_auth_type(tx: *const Transaction) -> HtpAuthType {
tx.as_ref()
.map(|tx| tx.request_auth_type)
.unwrap_or(HtpAuthType::ERROR)
}
/// Get a transaction's request hostname.
///
/// tx: Transaction pointer.
///
/// Returns the request hostname or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_hostname(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.request_hostname.as_ref())
.map(|hostname| hostname as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the transaction's request port number.
///
/// tx: Transaction pointer.
///
/// Returns the request port number or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_port_number(tx: *const Transaction) -> i32 {
tx.as_ref()
.and_then(|tx| tx.request_port_number.as_ref())
.map(|port| *port as i32)
.unwrap_or(-1)
}
/// Get a transaction's request message length.
///
/// tx: Transaction pointer.
///
/// Returns the request message length or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_message_len(tx: *const Transaction) -> i64 {
tx.as_ref()
.map(|tx| tx.request_message_len.try_into().ok().unwrap_or(-1))
.unwrap_or(-1)
}
/// Get a transaction's response line.
///
/// tx: Transaction pointer.
///
/// Returns the response line or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_line(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.response_line.as_ref())
.map(|response_line| response_line as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's response protocol.
///
/// tx: Transaction pointer.
///
/// Returns the response protocol or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_protocol(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.response_protocol.as_ref())
.map(|response_protocol| response_protocol as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's response protocol number.
///
/// tx: Transaction pointer.
///
/// Returns the protocol number or ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_protocol_number(tx: *const Transaction) -> HtpProtocol {
tx.as_ref()
.map(|tx| tx.response_protocol_number)
.unwrap_or(HtpProtocol::Error)
}
/// Get the transaction's response status.
///
/// tx: Transaction pointer.
///
/// Returns the response status or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_status(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.response_status.as_ref())
.map(|response_status| response_status as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the transaction's response status number.
///
/// tx: Transaction pointer.
///
/// Returns the response status number or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_status_number(tx: *const Transaction) -> i32 {
tx.as_ref()
.map(|tx| match tx.response_status_number {
HtpResponseNumber::Unknown => 0,
HtpResponseNumber::Invalid => -1,
HtpResponseNumber::Valid(status) => status as i32,
})
.unwrap_or(-1)
}
/// Get a transaction's response message.
///
/// tx: Transaction pointer.
///
/// Returns the response message or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_message(tx: *const Transaction) -> *const Bstr {
tx.as_ref()
.and_then(|tx| tx.response_message.as_ref())
.map(|response_message| response_message as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get a transaction's response headers.
///
/// tx: Transaction pointer.
///
/// Returns the response headers or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_headers(tx: *const Transaction) -> *const Headers {
tx.as_ref()
.map(|tx| &tx.response_headers as *const Headers)
.unwrap_or(std::ptr::null())
}
/// Get the first response header value matching the key from a transaction.
///
/// tx: Transaction pointer.
/// ckey: Header name to match.
///
/// Returns the header or NULL when not found or on error
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_header(
tx: *const Transaction, ckey: *const libc::c_char,
) -> *const Header {
tx.as_ref()
.map(|tx| htp_headers_get(&tx.response_headers, ckey))
.unwrap_or(std::ptr::null())
}
/// Get a transaction's response message length.
///
/// tx: Transaction pointer.
///
/// Returns the response message length or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_message_len(tx: *const Transaction) -> i64 {
tx.as_ref()
.map(|tx| tx.response_message_len.try_into().ok().unwrap_or(-1))
.unwrap_or(-1)
}
/// Get the transaction's bit flags.
///
/// tx: Transaction pointer.
///
/// Returns the flags represented as an integer or 0 if the flags are empty
/// or a NULL ptr is passed as an argument.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_flags(tx: *const Transaction) -> u64 {
tx.as_ref().map(|tx| tx.flags).unwrap_or(0)
}
/// Get the transaction's request progress.
///
/// tx: Transaction pointer.
///
/// Returns the progress or HTP_REQUEST_ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_request_progress(tx: *const Transaction) -> HtpRequestProgress {
tx.as_ref()
.map(|tx| tx.request_progress)
.unwrap_or(HtpRequestProgress::ERROR)
}
/// Get the transaction's response progress.
///
/// tx: Transaction pointer.
///
/// Returns the progress or ERROR on error.
/// # Safety
/// When calling this method, you have to ensure that tx is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_response_progress(tx: *const Transaction) -> HtpResponseProgress {
tx.as_ref()
.map(|tx| tx.response_progress)
.unwrap_or(HtpResponseProgress::ERROR)
}
/// Get the data's transaction.
///
/// Returns the transaction or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that data is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_data_tx(data: *const Data) -> *const Transaction {
data.as_ref()
.map(|data| data.tx() as *const Transaction)
.unwrap_or(std::ptr::null())
}
/// Get the data pointer.
///
/// Returns the data or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that data is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_data_data(data: *const Data) -> *const u8 {
data.as_ref()
.map(|data| data.data())
.unwrap_or(std::ptr::null())
}
/// Get the length of the data.
///
/// Returns the length or -1 on error.
/// # Safety
/// When calling this method, you have to ensure that data is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_data_len(data: *const Data) -> isize {
data.as_ref()
.map(|data| isize::try_from(data.len()).unwrap_or(-1))
.unwrap_or(-1)
}
/// Get whether this data is empty.
///
/// Returns true if data is NULL or zero-length.
/// # Safety
/// When calling this method, you have to ensure that data is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_tx_data_is_empty(data: *const Data) -> bool {
data.as_ref().map(|data| data.is_empty()).unwrap_or(true)
}
rust/htp/src/c_api/uri.rs
+27
View File
@@ -0,0 +1,27 @@
use crate::{bstr::Bstr, uri::Uri};
/// Get the hostname of a uri.
///
/// Returns the hostname for uri or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that uri is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_uri_hostname(uri: *const Uri) -> *const Bstr {
uri.as_ref()
.and_then(|uri| uri.hostname.as_ref())
.map(|hostname| hostname as *const Bstr)
.unwrap_or(std::ptr::null())
}
/// Get the path of a uri.
///
/// Returns the path for uri or NULL on error.
/// # Safety
/// When calling this method, you have to ensure that uri is either properly initialized or NULL
#[no_mangle]
pub unsafe extern "C" fn htp_uri_path(uri: *const Uri) -> *const Bstr {
uri.as_ref()
.and_then(|uri| uri.path.as_ref())
.map(|path| path as *const Bstr)
.unwrap_or(std::ptr::null())
}
rust/htp/src/config.rs
+575
View File
@@ -0,0 +1,575 @@
use crate::decompressors::Options;
use crate::{
error::Result,
hook::{DataHook, TxHook},
unicode_bestfit_map::UnicodeBestfitMap,
HtpStatus,
};
#[cfg(test)]
use crate::hook::{DataNativeCallbackFn, TxNativeCallbackFn};
/// Configuration for libhtp parsing.
#[derive(Clone)]
pub struct Config {
/// The maximum size of the buffer that is used when the current
/// input chunk does not contain all the necessary data (e.g., a header
/// line that spans several packets).
pub(crate) field_limit: usize,
/// Server personality identifier.
pub(crate) server_personality: HtpServerPersonality,
/// Decoder configuration for url path.
pub(crate) decoder_cfg: DecoderConfig,
/// Request start hook, invoked when the parser receives the first byte of a new
/// request. Because an HTTP transaction always starts with a request, this hook
/// doubles as a transaction start hook.
pub(crate) hook_request_start: TxHook,
/// Request line hook, invoked after a request line has been parsed.
pub(crate) hook_request_line: TxHook,
/// Receives raw request header data, starting immediately after the request line,
/// including all headers as they are seen on the TCP connection, and including the
/// terminating empty line. Not available on genuine HTTP/0.9 requests (because
/// they don't use headers).
pub(crate) hook_request_header_data: DataHook,
/// Request headers hook, invoked after all request headers are seen.
#[cfg(test)]
pub(crate) hook_request_headers: TxHook,
/// Request body data hook, invoked every time body data is available. Each
/// invocation will provide a Data instance. Chunked data
/// will be dechunked before the data is passed to this hook. Decompression
/// is not currently implemented. At the end of the request body
/// there will be a call with the data set to None.
pub(crate) hook_request_body_data: DataHook,
/// Receives raw request trailer data, which can be available on requests that have
/// chunked bodies. The data starts immediately after the zero-length chunk
/// and includes the terminating empty line.
pub(crate) hook_request_trailer_data: DataHook,
/// Request trailer hook, invoked after all trailer headers are seen,
/// and if they are seen (not invoked otherwise).
pub(crate) hook_request_trailer: TxHook,
/// Request hook, invoked after a complete request is seen.
pub(crate) hook_request_complete: TxHook,
/// Response startup hook, invoked when a response transaction is found and
/// processing started.
pub(crate) hook_response_start: TxHook,
/// Response line hook, invoked after a response line has been parsed.
#[cfg(test)]
pub(crate) hook_response_line: TxHook,
/// Receives raw response header data, starting immediately after the status line
/// and including all headers as they are seen on the TCP connection, and including the
/// terminating empty line. Not available on genuine HTTP/0.9 responses (because
/// they don't have response headers).
pub(crate) hook_response_header_data: DataHook,
/// Response headers book, invoked after all response headers have been seen.
#[cfg(test)]
pub(crate) hook_response_headers: TxHook,
/// Response body data hook, invoked every time body data is available. Each
/// invocation will provide a Data instance. Chunked data
/// will be dechunked before the data is passed to this hook. By default,
/// compressed data will be decompressed, but decompression can be disabled
/// in configuration. At the end of the response body there will be a call
/// with the data pointer set to NULL.
pub(crate) hook_response_body_data: DataHook,
/// Receives raw response trailer data, which can be available on responses that have
/// chunked bodies. The data starts immediately after the zero-length chunk
/// and includes the terminating empty line.
pub(crate) hook_response_trailer_data: DataHook,
/// Response trailer hook, invoked after all trailer headers have been processed,
/// and only if the trailer exists.
pub(crate) hook_response_trailer: TxHook,
/// Response hook, invoked after a response has been seen. Because sometimes servers
/// respond before receiving complete requests, a response_complete callback may be
/// invoked prior to a request_complete callback.
pub(crate) hook_response_complete: TxHook,
/// Transaction complete hook, which is invoked once the entire transaction is
/// considered complete (request and response are both complete). This is always
/// the last hook to be invoked.
#[cfg(test)]
pub(crate) hook_transaction_complete: TxHook,
/// Reaction to leading whitespace on the request line
pub(crate) requestline_leading_whitespace_unwanted: HtpUnwanted,
/// Whether to decompress compressed request bodies.
pub(crate) request_decompression_enabled: bool,
/// Configuration options for decompression.
pub(crate) compression_options: Options,
/// Maximum number of transactions
pub(crate) max_tx: u32,
/// Maximum number of headers
pub(crate) number_headers_limit: u32,
}
impl Default for Config {
fn default() -> Self {
Self {
field_limit: 18000,
server_personality: HtpServerPersonality::MINIMAL,
decoder_cfg: Default::default(),
hook_request_start: TxHook::default(),
hook_request_line: TxHook::default(),
hook_request_header_data: DataHook::default(),
#[cfg(test)]
hook_request_headers: TxHook::default(),
hook_request_body_data: DataHook::default(),
hook_request_trailer_data: DataHook::default(),
hook_request_trailer: TxHook::default(),
hook_request_complete: TxHook::default(),
hook_response_start: TxHook::default(),
#[cfg(test)]
hook_response_line: TxHook::default(),
hook_response_header_data: DataHook::default(),
#[cfg(test)]
hook_response_headers: TxHook::default(),
hook_response_body_data: DataHook::default(),
hook_response_trailer_data: DataHook::default(),
hook_response_trailer: TxHook::default(),
hook_response_complete: TxHook::default(),
#[cfg(test)]
hook_transaction_complete: TxHook::default(),
requestline_leading_whitespace_unwanted: HtpUnwanted::Ignore,
request_decompression_enabled: false,
compression_options: Options::default(),
max_tx: 512,
number_headers_limit: 1024,
}
}
}
/// Configuration options for decoding.
#[derive(Copy, Clone)]
pub(crate) struct DecoderConfig {
///Whether to double decode the path in normalized uri
pub(crate) double_decode_normalized_path: bool,
/// Whether to double decode the query in the normalized uri
pub(crate) double_decode_normalized_query: bool,
// Path-specific decoding options.
/// Convert backslash characters to slashes.
pub(crate) backslash_convert_slashes: bool,
/// Convert to lowercase.
pub(crate) convert_lowercase: bool,
/// Compress slash characters.
pub(crate) path_separators_compress: bool,
/// Should we URL-decode encoded path segment separators?
pub(crate) path_separators_decode: bool,
/// Should we decode '+' characters to spaces?
pub(crate) plusspace_decode: bool,
// Special characters options.
/// Controls how raw NUL bytes are handled.
pub(crate) nul_raw_terminates: bool,
/// Determines server response to a raw NUL byte in the path.
pub(crate) nul_raw_unwanted: HtpUnwanted,
/// Reaction to control characters.
pub(crate) control_chars_unwanted: HtpUnwanted,
/// Allow whitespace characters in request uri path
pub(crate) allow_space_uri: bool,
// URL encoding options.
/// Should we decode %u-encoded characters?
pub(crate) u_encoding_decode: bool,
/// Reaction to %u encoding.
pub(crate) u_encoding_unwanted: HtpUnwanted,
/// Handling of invalid URL encodings.
pub(crate) url_encoding_invalid_handling: HtpUrlEncodingHandling,
/// Reaction to invalid URL encoding.
pub(crate) url_encoding_invalid_unwanted: HtpUnwanted,
/// Controls how encoded NUL bytes are handled.
pub(crate) nul_encoded_terminates: bool,
/// How are we expected to react to an encoded NUL byte?
pub(crate) nul_encoded_unwanted: HtpUnwanted,
// Normalized URI preference
/// Controls whether the client wants the complete or partial normalized URI.
pub(crate) normalized_uri_include_all: bool,
// UTF-8 options.
/// Controls how invalid UTF-8 characters are handled.
pub(crate) utf8_invalid_unwanted: HtpUnwanted,
/// Convert UTF-8 characters into bytes using best-fit mapping.
pub(crate) utf8_convert_bestfit: bool,
/// Best-fit map for UTF-8 decoding.
pub(crate) bestfit_map: UnicodeBestfitMap,
}
impl Default for DecoderConfig {
fn default() -> Self {
Self {
double_decode_normalized_path: false,
double_decode_normalized_query: false,
backslash_convert_slashes: false,
convert_lowercase: false,
path_separators_compress: false,
path_separators_decode: false,
plusspace_decode: true,
nul_raw_terminates: false,
nul_raw_unwanted: HtpUnwanted::Ignore,
control_chars_unwanted: HtpUnwanted::Ignore,
allow_space_uri: false,
u_encoding_decode: false,
u_encoding_unwanted: HtpUnwanted::Ignore,
url_encoding_invalid_handling: HtpUrlEncodingHandling::PRESERVE_PERCENT,
url_encoding_invalid_unwanted: HtpUnwanted::Ignore,
nul_encoded_terminates: false,
nul_encoded_unwanted: HtpUnwanted::Ignore,
normalized_uri_include_all: false,
utf8_invalid_unwanted: HtpUnwanted::Ignore,
utf8_convert_bestfit: false,
bestfit_map: UnicodeBestfitMap::default(),
}
}
}
/// Enumerates the possible server personalities.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum HtpServerPersonality {
/// Minimal personality that performs as little work as possible. All optional
/// features are disabled. This personality is a good starting point for customization.
MINIMAL,
/// A generic personality that aims to work reasonably well for all server types.
GENERIC,
/// The IDS personality tries to perform as much decoding as possible.
IDS,
/// Mimics the behavior of IIS 4.0, as shipped with Windows NT 4.0.
IIS_4_0,
/// Mimics the behavior of IIS 5.0, as shipped with Windows 2000.
IIS_5_0,
/// Mimics the behavior of IIS 5.1, as shipped with Windows XP Professional.
IIS_5_1,
/// Mimics the behavior of IIS 6.0, as shipped with Windows 2003.
IIS_6_0,
/// Mimics the behavior of IIS 7.0, as shipped with Windows 2008.
IIS_7_0,
/// Mimics the behavior of IIS 7.5, as shipped with Windows 7.
IIS_7_5,
/// Mimics the behavior of Apache 2.x.
APACHE_2,
}
/// Enumerates the ways in which servers respond to malformed data.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub(crate) enum HtpUnwanted {
/// Ignores problem.
Ignore,
/// Responds with HTTP 400 status code.
Code400 = 400,
}
/// Enumerates the possible approaches to handling invalid URL-encodings.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum HtpUrlEncodingHandling {
/// Ignore invalid URL encodings and leave the % in the data.
PRESERVE_PERCENT,
/// Ignore invalid URL encodings, but remove the % from the data.
REMOVE_PERCENT,
/// Decode invalid URL encodings.
PROCESS_INVALID,
}
impl Config {
/// Registers a request_complete callback, which is invoked when we see the
/// first bytes of data from a request.
#[cfg(test)]
pub(crate) fn register_request_complete(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_request_complete.register(cbk_fn);
}
/// Registers a request_body_data callback, which is invoked whenever we see
/// bytes of request body data.
#[cfg(test)]
pub(crate) fn register_request_body_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_request_body_data.register(cbk_fn);
}
/// Registers a request_header_data callback, which is invoked when we see header
/// data. This callback receives raw header data as seen on the connection, including
/// the terminating line and anything seen after the request line.
#[cfg(test)]
pub(crate) fn register_request_header_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_request_header_data.register(cbk_fn);
}
/// Registers a request_headers callback, which is invoked after we see all the
/// request headers.
#[cfg(test)]
pub(crate) fn register_request_headers(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_request_headers.register(cbk_fn);
}
/// Registers a request_line callback, which is invoked after we parse the entire
/// request line.
#[cfg(test)]
pub(crate) fn register_request_line(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_request_line.register(cbk_fn);
}
/// Registers a request_start callback, which is invoked every time a new
/// request begins and before any parsing is done.
#[cfg(test)]
pub(crate) fn register_request_start(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_request_start.register(cbk_fn);
}
/// Registers a request_trailer_data callback, which may be invoked on requests with
/// chunked bodies. This callback receives the raw response trailer data after the zero-length
/// chunk including the terminating line.
#[cfg(test)]
pub(crate) fn register_request_trailer_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_request_trailer_data.register(cbk_fn);
}
/// Registers a response_body_data callback, which is invoked whenever we see
/// bytes of response body data.
#[cfg(test)]
pub(crate) fn register_response_body_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_response_body_data.register(cbk_fn);
}
/// Registers a response_complete callback, which is invoked when we see the
/// first bytes of data from a response.
#[cfg(test)]
pub(crate) fn register_response_complete(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_response_complete.register(cbk_fn);
}
/// Registers a response_header_data callback, which is invoked when we see header
/// data. This callback receives raw header data as seen on the connection, including
/// the terminating line and anything seen after the response line.
#[cfg(test)]
pub(crate) fn register_response_header_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_response_header_data.register(cbk_fn);
}
/// Registers a response_headers callback, which is invoked after we see all the
/// response headers.
#[cfg(test)]
pub(crate) fn register_response_headers(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_response_headers.register(cbk_fn);
}
/// Registers a response_line callback, which is invoked after we parse the entire
/// response line.
#[cfg(test)]
pub(crate) fn register_response_line(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_response_line.register(cbk_fn);
}
/// Registers a response_start callback, which is invoked when we see the
/// first bytes of data from a response.
#[cfg(test)]
pub(crate) fn register_response_start(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_response_start.register(cbk_fn);
}
/// Registers a response_trailer_data callback, which may be invoked on responses with
/// chunked bodies. This callback receives the raw response trailer data after the zero-length
/// chunk and including the terminating line.
#[cfg(test)]
pub(crate) fn register_response_trailer_data(&mut self, cbk_fn: DataNativeCallbackFn) {
self.hook_response_trailer_data.register(cbk_fn);
}
/// Registers a transaction_complete callback, which is invoked once the request and response
/// are both complete.
#[cfg(test)]
pub(crate) fn register_transaction_complete(&mut self, cbk_fn: TxNativeCallbackFn) {
self.hook_transaction_complete.register(cbk_fn);
}
/// Enable or disable the double decoding of the path in the normalized uri
pub(crate) fn set_double_decode_normalized_path(
&mut self, double_decode_normalized_path: bool,
) {
self.decoder_cfg.double_decode_normalized_path = double_decode_normalized_path;
}
/// Enable or disable the double decoding of the query in the normalized uri
pub(crate) fn set_double_decode_normalized_query(
&mut self, double_decode_normalized_query: bool,
) {
self.decoder_cfg.double_decode_normalized_query = double_decode_normalized_query;
}
/// Configures the maximum size of the buffer LibHTP will use when all data is not available
/// in the current buffer (e.g., a very long header line that might span several packets). This
/// limit is controlled by the field_limit parameter.
pub(crate) fn set_field_limit(&mut self, field_limit: usize) {
self.field_limit = field_limit;
}
/// Enable or disable spaces in URIs. Disabled by default.
pub(crate) fn set_allow_space_uri(&mut self, allow_space: bool) {
self.decoder_cfg.allow_space_uri = allow_space;
}
/// Configure desired server personality.
/// Returns an Error if the personality is not supported.
pub(crate) fn set_server_personality(
&mut self, personality: HtpServerPersonality,
) -> Result<()> {
match personality {
HtpServerPersonality::MINIMAL => {}
HtpServerPersonality::GENERIC => {
self.set_backslash_convert_slashes(true);
self.set_path_separators_decode(true);
self.set_path_separators_compress(true);
}
HtpServerPersonality::IDS => {
self.set_backslash_convert_slashes(true);
self.set_path_separators_decode(true);
self.set_path_separators_compress(true);
self.set_convert_lowercase(true);
self.set_utf8_convert_bestfit(true);
self.set_u_encoding_decode(true);
self.set_requestline_leading_whitespace_unwanted(HtpUnwanted::Ignore);
}
HtpServerPersonality::APACHE_2 => {
self.set_backslash_convert_slashes(false);
self.set_path_separators_decode(false);
self.set_path_separators_compress(true);
self.set_u_encoding_decode(false);
self.set_url_encoding_invalid_handling(HtpUrlEncodingHandling::PRESERVE_PERCENT);
self.set_url_encoding_invalid_unwanted(HtpUnwanted::Code400);
self.set_control_chars_unwanted(HtpUnwanted::Ignore);
self.set_requestline_leading_whitespace_unwanted(HtpUnwanted::Code400);
}
HtpServerPersonality::IIS_5_1 => {
self.set_backslash_convert_slashes(true);
self.set_path_separators_decode(true);
self.set_path_separators_compress(true);
self.set_u_encoding_decode(false);
self.set_url_encoding_invalid_handling(HtpUrlEncodingHandling::PRESERVE_PERCENT);
self.set_control_chars_unwanted(HtpUnwanted::Ignore);
self.set_requestline_leading_whitespace_unwanted(HtpUnwanted::Ignore);
}
HtpServerPersonality::IIS_6_0 => {
self.set_backslash_convert_slashes(true);
self.set_path_separators_decode(true);
self.set_path_separators_compress(true);
self.set_u_encoding_decode(true);
self.set_url_encoding_invalid_handling(HtpUrlEncodingHandling::PRESERVE_PERCENT);
self.set_u_encoding_unwanted(HtpUnwanted::Code400);
self.set_control_chars_unwanted(HtpUnwanted::Code400);
self.set_requestline_leading_whitespace_unwanted(HtpUnwanted::Ignore);
}
HtpServerPersonality::IIS_7_0 | HtpServerPersonality::IIS_7_5 => {
self.set_backslash_convert_slashes(true);
self.set_path_separators_decode(true);
self.set_path_separators_compress(true);
self.set_u_encoding_decode(true);
self.set_url_encoding_invalid_handling(HtpUrlEncodingHandling::PRESERVE_PERCENT);
self.set_url_encoding_invalid_unwanted(HtpUnwanted::Code400);
self.set_control_chars_unwanted(HtpUnwanted::Code400);
self.set_requestline_leading_whitespace_unwanted(HtpUnwanted::Ignore);
}
_ => return Err(HtpStatus::ERROR),
}
// Remember the personality
self.server_personality = personality;
Ok(())
}
/// Sets the replacement character that will be used in the lossy best-fit
/// mapping from multi-byte to single-byte streams. The question mark character
/// is used as the default replacement byte.
pub(crate) fn set_bestfit_replacement_byte(&mut self, b: u8) {
self.decoder_cfg.bestfit_map.replacement_byte = b;
}
/// Configures how the server handles to invalid URL encoding.
pub(crate) fn set_url_encoding_invalid_handling(&mut self, handling: HtpUrlEncodingHandling) {
self.decoder_cfg.url_encoding_invalid_handling = handling;
}
/// Configures the handling of raw NUL bytes. If enabled, raw NUL terminates strings.
pub(crate) fn set_nul_raw_terminates(&mut self, enabled: bool) {
self.decoder_cfg.nul_raw_terminates = enabled;
}
/// Configures how the server reacts to encoded NUL bytes. Some servers will stop at
/// at NUL, while some will respond with 400 or 404. When the termination option is not
/// used, the NUL byte will remain in the path.
pub(crate) fn set_nul_encoded_terminates(&mut self, enabled: bool) {
self.decoder_cfg.nul_encoded_terminates = enabled;
}
/// Configures whether %u-encoded sequences are decoded. Such sequences
/// will be treated as invalid URL encoding if decoding is not desirable.
pub(crate) fn set_u_encoding_decode(&mut self, enabled: bool) {
self.decoder_cfg.u_encoding_decode = enabled;
}
/// Configures whether backslash characters are treated as path segment separators. They
/// are not on Unix systems, but are on Windows systems. If this setting is enabled, a path
/// such as "/one\two/three" will be converted to "/one/two/three".
pub(crate) fn set_backslash_convert_slashes(&mut self, enabled: bool) {
self.decoder_cfg.backslash_convert_slashes = enabled;
}
/// Configures whether encoded path segment separators will be decoded. Apache does not do
/// this by default, but IIS does. If enabled, a path such as "/one%2ftwo" will be normalized
/// to "/one/two". If the backslash_separators option is also enabled, encoded backslash
/// characters will be converted too (and subsequently normalized to forward slashes).
pub(crate) fn set_path_separators_decode(&mut self, enabled: bool) {
self.decoder_cfg.path_separators_decode = enabled;
}
/// Configures whether consecutive path segment separators will be compressed. When enabled, a path
/// such as "/one//two" will be normalized to "/one/two". Backslash conversion and path segment separator
/// decoding are carried out before compression. For example, the path "/one\\/two\/%5cthree/%2f//four"
/// will be converted to "/one/two/three/four" (assuming all 3 options are enabled).
pub(crate) fn set_path_separators_compress(&mut self, enabled: bool) {
self.decoder_cfg.path_separators_compress = enabled;
}
/// Configures whether plus characters are converted to spaces when decoding URL-encoded strings. This
/// is appropriate to do for parameters, but not for URLs. Only applies to contexts where decoding
/// is taking place.
pub(crate) fn set_plusspace_decode(&mut self, enabled: bool) {
self.decoder_cfg.plusspace_decode = enabled;
}
/// Configures whether input data will be converted to lowercase. Useful for handling servers with
/// case-insensitive filesystems.
pub(crate) fn set_convert_lowercase(&mut self, enabled: bool) {
self.decoder_cfg.convert_lowercase = enabled;
}
/// Controls whether the data should be treated as UTF-8 and converted to a single-byte
/// stream using best-fit mapping.
pub(crate) fn set_utf8_convert_bestfit(&mut self, enabled: bool) {
self.decoder_cfg.utf8_convert_bestfit = enabled;
}
/// Configures reaction to %u-encoded sequences in input data.
pub(crate) fn set_u_encoding_unwanted(&mut self, unwanted: HtpUnwanted) {
self.decoder_cfg.u_encoding_unwanted = unwanted;
}
/// Controls reaction to raw control characters in the data.
pub(crate) fn set_control_chars_unwanted(&mut self, unwanted: HtpUnwanted) {
self.decoder_cfg.control_chars_unwanted = unwanted;
}
/// Controls whether to use complete or partial URI normalization
pub(crate) fn set_normalized_uri_include_all(&mut self, set: bool) {
self.decoder_cfg.normalized_uri_include_all = set;
}
/// Configures how the server reacts to invalid URL encoding.
pub(crate) fn set_url_encoding_invalid_unwanted(&mut self, unwanted: HtpUnwanted) {
self.decoder_cfg.url_encoding_invalid_unwanted = unwanted;
}
/// Configures how the server reacts to leading whitespace on the request line.
pub(crate) fn set_requestline_leading_whitespace_unwanted(&mut self, unwanted: HtpUnwanted) {
self.requestline_leading_whitespace_unwanted = unwanted;
}
/// Configures whether request data is decompressed.
pub(crate) fn set_request_decompression(&mut self, set: bool) {
self.request_decompression_enabled = set;
}
/// Configures many layers of compression we try to decompress.
pub(crate) fn set_decompression_layer_limit(&mut self, limit: Option<u32>) {
self.compression_options.set_layer_limit(limit);
}
}
rust/htp/src/connection.rs
+128
View File
@@ -0,0 +1,128 @@
use crate::log::Log;
use std::{cell::RefCell, collections::VecDeque, net::IpAddr, rc::Rc, time::SystemTime};
use time::OffsetDateTime;
/// Export Connection ConnectionFlags
#[repr(C)]
pub(crate) struct ConnectionFlags;
/// `Connection` Flags
impl ConnectionFlags {
/// Seen pipelined requests.
pub(crate) const PIPELINED: u8 = 0x01;
/// Seen extra data after a HTTP 0.9 communication.
pub(crate) const HTTP_0_9_EXTRA: u8 = 0x02;
}
/// Stores information about the session.
pub struct Connection {
/// Client IP address.
pub(crate) client_addr: Option<IpAddr>,
/// Client port.
pub(crate) client_port: Option<u16>,
/// Server IP address.
pub(crate) server_addr: Option<IpAddr>,
/// Server port.
pub(crate) server_port: Option<u16>,
/// Messages channel associated with this connection.
log_channel: Rc<RefCell<VecDeque<Log>>>,
/// Parsing flags.
pub(crate) flags: u8,
/// When was this connection opened?
pub(crate) open_timestamp: OffsetDateTime,
/// When was this connection closed?
pub(crate) close_timestamp: OffsetDateTime,
/// Inbound data counter.
pub(crate) request_data_counter: u64,
/// Outbound data counter.
pub(crate) response_data_counter: u64,
}
impl Default for Connection {
/// Returns a new Connection instance with default values.
fn default() -> Self {
Self {
client_addr: None,
client_port: None,
server_addr: None,
server_port: None,
log_channel: Rc::new(RefCell::new(VecDeque::new())),
flags: 0,
open_timestamp: OffsetDateTime::from(SystemTime::now()),
close_timestamp: OffsetDateTime::from(SystemTime::now()),
request_data_counter: 0,
response_data_counter: 0,
}
}
}
impl Connection {
/// Opens a connection. This function will essentially only store the provided data
/// for future reference.
pub(crate) fn open(
&mut self, client_addr: Option<IpAddr>, client_port: Option<u16>,
server_addr: Option<IpAddr>, server_port: Option<u16>, timestamp: Option<OffsetDateTime>,
) {
self.client_addr = client_addr;
self.client_port = client_port;
self.server_addr = server_addr;
self.server_port = server_port;
// Remember when the connection was opened.
if let Some(timestamp) = timestamp {
self.open_timestamp = timestamp;
}
}
/// Closes the connection.
pub(crate) fn close(&mut self, timestamp: Option<OffsetDateTime>) {
// Update timestamp.
if let Some(timestamp) = timestamp {
self.close_timestamp = timestamp;
}
}
/// Keeps track of inbound packets and data.
pub(crate) fn track_inbound_data(&mut self, len: usize) {
self.request_data_counter = (self.request_data_counter).wrapping_add(len as u64);
}
/// Keeps track of outbound packets and data.
pub(crate) fn track_outbound_data(&mut self, len: usize) {
self.response_data_counter = (self.response_data_counter).wrapping_add(len as u64);
}
/// Return the log channel sender
pub(crate) fn get_sender(&self) -> &Rc<RefCell<VecDeque<Log>>> {
&self.log_channel
}
/// Drains and returns a vector of all current logs received by the log channel
#[cfg(test)]
pub(crate) fn get_logs(&self) -> Vec<Log> {
let mut lc = self.log_channel.borrow_mut();
let mut r = Vec::with_capacity(lc.len());
while let Some(e) = lc.pop_front() {
r.push(e)
}
r
}
/// Returns the next logged message received by the log channel
pub(crate) fn get_next_log(&self) -> Option<Log> {
let mut lc = self.log_channel.borrow_mut();
lc.pop_front()
}
}
impl PartialEq for Connection {
/// Returns true if connections are the same, false otherwise.
fn eq(&self, rhs: &Self) -> bool {
self.client_addr == rhs.client_addr
&& self.client_port == rhs.client_port
&& self.server_addr == rhs.server_addr
&& self.server_port == rhs.server_port
}
}
rust/htp/src/connection_parser.rs
+946
View File
@@ -0,0 +1,946 @@
use crate::{
bstr::Bstr,
config::Config,
connection::{Connection, ConnectionFlags},
decompressors::HtpContentEncoding,
error::Result,
hook::DataHook,
log::Logger,
transaction::{HtpRequestProgress, HtpResponseProgress, HtpTransferCoding, Transaction},
transactions::Transactions,
util::{FlagOperations, HtpFlags},
HtpStatus,
};
use std::{any::Any, borrow::Cow, cell::Cell, net::IpAddr, time::SystemTime};
use time::OffsetDateTime;
/// Enumerates parsing state.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
pub(crate) enum State {
/// Default state.
None,
/// State once a transaction is processed or about to be processed.
Idle,
/// State for request/response line parsing.
Line,
/// State for header parsing.
Headers,
/// State for finalizing chunked body data parsing.
BodyChunkedDataEnd,
/// State for chunked body data.
BodyChunkedData,
/// Parse the chunked length state.
BodyChunkedLength,
/// State to determine encoding of body data.
BodyDetermine,
/// State for finalizing transaction side.
Finalize,
// Used by request_state only
/// State for determining the request protocol.
Protocol,
/// State to determine if there is a CONNECT request.
ConnectCheck,
/// State to determine if inbound parsing needs to be suspended.
ConnectProbeData,
/// State to determine if inbound parsing can continue if it was suspended.
ConnectWaitResponse,
/// State to process request body data.
BodyIdentity,
/// State to consume remaining data in request buffer for the HTTP 0.9 case.
IgnoreDataAfterHTTP09,
// Used by response_state only
/// State to consume response remaining body data when content-length is unknown.
BodyIdentityStreamClose,
/// State to consume response body data when content-length is known.
BodyIdentityCLKnown,
}
/// Enumerates all stream states. Each connection has two streams, one
/// inbound and one outbound. Their states are tracked separately.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum HtpStreamState {
/// Default stream state.
NEW,
/// State when connection is open.
OPEN,
/// State when connection is closed.
CLOSED,
/// State when stream produces a fatal error.
ERROR,
/// State for a tunnelled stream.
TUNNEL,
/// State when parsing is suspended and not consumed in order. This is to
/// allow processing on another stream.
DATA_OTHER,
/// State when we should stop parsing the associated connection.
STOP,
/// State when all current data in the stream has been processed.
DATA,
}
#[derive(Debug, Default, Clone)]
/// This structure is used to pass data (for example
/// request and response body buffers or gaps) to parsers.
pub(crate) struct ParserData<'a> {
/// Ref to the data buffer.
data: Option<Cow<'a, [u8]>>,
// Length of data gap. Only set if is a gap.
gap_len: Option<usize>,
// Current position offset of the data to parse
position: Cell<usize>,
// Current callback data position
callback_position: usize,
}
impl ParserData<'_> {
/// Returns a pointer to the raw data associated with Data.
/// This returns a pointer to the entire data chunk.
pub(crate) fn data_ptr(&self) -> *const u8 {
self.data()
.as_ref()
.map(|data| data.as_ptr())
.unwrap_or(std::ptr::null())
}
/// Returns the unconsumed data
pub(crate) fn data(&self) -> Option<&[u8]> {
let data = self.data.as_ref()?;
if self.position.get() <= data.len() {
Some(&data[self.position.get()..])
} else {
None
}
}
/// Returns the length of the unconsumed data.
pub(crate) fn len(&self) -> usize {
if let Some(gap_len) = self.gap_len {
if self.position.get() >= gap_len {
0
} else {
gap_len - self.position.get()
}
} else {
self.as_slice().len()
}
}
/// Returns how much data has been consumed so far
fn consumed_len(&self) -> usize {
self.position.get()
}
/// Return an immutable slice view of the unconsumed data.
pub(crate) fn as_slice(&self) -> &[u8] {
if let Some(data) = self.data.as_ref() {
if self.position.get() <= data.len() {
return &data[self.position.get()..];
}
}
b""
}
/// Determines if this chunk is a gap or not
pub(crate) fn is_gap(&self) -> bool {
self.gap_len.is_some()
}
/// Determine whether there is no more data to consume.
pub(crate) fn is_empty(&self) -> bool {
self.len() == 0
}
/// Set the position offset into the data for parsing
fn set_position(&self, position: usize) {
self.position.set(position);
}
/// Advances the internal position where we are parsing
pub(crate) fn consume(&self, consumed: usize) {
self.set_position(self.position.get() + consumed);
}
/// Decrements the internal position where we are parsing
fn unconsume(&self, unconsume: usize) {
if unconsume < self.position.get() {
self.set_position(self.position.get() - unconsume);
} else {
self.set_position(0);
}
}
/// Make an owned version of this data.
#[cfg(test)]
pub(crate) fn into_owned(self) -> ParserData<'static> {
ParserData {
data: self.data.map(|d| Cow::Owned(d.into_owned())),
gap_len: self.gap_len,
position: self.position,
callback_position: self.callback_position,
}
}
/// Callback data is raw data buffer content that is passed to the
/// application via the header and trailer data hooks.
///
/// This function will return any data that has been consumed but not
/// yet returned from this function.
pub(crate) fn callback_data(&mut self) -> &[u8] {
if let Some(data) = self.data.as_ref() {
if self.position.get() <= data.len() && self.callback_position <= self.position.get() {
let d = &data[self.callback_position..self.position.get()];
self.callback_position = self.position.get();
return d;
}
}
b""
}
/// Sets the callback start location to the current parsing location
pub(crate) fn reset_callback_start(&mut self) {
self.callback_position = self.position.get();
}
}
impl<'a> From<Option<&'a [u8]>> for ParserData<'a> {
fn from(data: Option<&'a [u8]>) -> Self {
ParserData {
data: data.map(Cow::Borrowed),
gap_len: None,
position: Cell::new(0),
callback_position: 0,
}
}
}
impl<'a> From<&'a [u8]> for ParserData<'a> {
fn from(data: &'a [u8]) -> Self {
ParserData {
data: Some(Cow::Borrowed(data)),
gap_len: None,
position: Cell::new(0),
callback_position: 0,
}
}
}
impl From<Vec<u8>> for ParserData<'static> {
fn from(data: Vec<u8>) -> Self {
ParserData {
data: Some(Cow::Owned(data)),
gap_len: None,
position: Cell::new(0),
callback_position: 0,
}
}
}
impl<'a> From<&'a Vec<u8>> for ParserData<'a> {
fn from(data: &'a Vec<u8>) -> Self {
ParserData {
data: Some(Cow::Borrowed(data.as_slice())),
gap_len: None,
position: Cell::new(0),
callback_position: 0,
}
}
}
impl From<usize> for ParserData<'_> {
fn from(gap_len: usize) -> Self {
ParserData {
data: None,
gap_len: Some(gap_len),
position: Cell::new(0),
callback_position: 0,
}
}
}
impl From<(*const u8, usize)> for ParserData<'_> {
fn from((data, len): (*const u8, usize)) -> Self {
if data.is_null() {
if len > 0 {
ParserData::from(len)
} else {
ParserData::from(b"".as_ref())
}
} else {
unsafe { ParserData::from(std::slice::from_raw_parts(data, len)) }
}
}
}
/// Stores information about the parsing process and associated transactions.
pub struct ConnectionParser {
// General fields
/// The logger structure associated with this parser
pub(crate) logger: Logger,
/// A reference to the current parser configuration structure.
pub(crate) cfg: &'static Config,
/// The connection structure associated with this parser.
pub(crate) conn: Connection,
/// Opaque user data associated with this parser.
pub(crate) user_data: Option<Box<dyn Any>>,
// Request parser fields
/// Parser inbound status. Starts as OK, but may turn into ERROR.
pub(crate) request_status: HtpStreamState,
/// Parser outbound status. Starts as OK, but may turn into ERROR.
pub(crate) response_status: HtpStreamState,
/// When true, this field indicates that there is unprocessed inbound data, and
/// that the response parsing code should stop at the end of the current request
/// in order to allow more requests to be produced.
pub(crate) response_data_other_at_tx_end: bool,
/// The time when the last request data chunk was received.
pub(crate) request_timestamp: OffsetDateTime,
/// How many bytes from the last input chunk have we consumed
/// This is mostly used from callbacks, where the caller
/// wants to know how far into the last chunk the parser is.
pub(crate) request_bytes_consumed: usize,
/// How many data chunks does the inbound connection stream consist of?
pub(crate) request_chunk_count: usize,
/// The index of the first chunk used in the current request.
pub(crate) request_chunk_request_index: usize,
/// Used to buffer a line of inbound data when buffering cannot be avoided.
pub(crate) request_buf: Bstr,
/// Stores the current value of a folded request header. Such headers span
/// multiple lines, and are processed only when all data is available.
pub(crate) request_header: Option<Bstr>,
/// The request body length declared in a valid request header. The key here
/// is "valid". This field will not be populated if the request contains both
/// a Transfer-Encoding header and a Content-Length header.
pub(crate) request_content_length: Option<u64>,
/// Holds the remaining request body length that we expect to read. This
/// field will be available only when the length of a request body is known
/// in advance, i.e. when request headers contain a Content-Length header.
pub(crate) request_body_data_left: Option<u64>,
/// Holds the amount of data that needs to be read from the
/// current data chunk. Only used with chunked request bodies.
pub(crate) request_chunked_length: Option<u64>,
/// Current request parser state.
pub(crate) request_state: State,
/// Previous request parser state. Used to detect state changes.
pub(crate) request_state_previous: State,
/// The hook that should be receiving raw connection data.
pub(crate) request_data_receiver_hook: Option<DataHook>,
// Response parser fields
/// The time when the last response data chunk was received.
pub(crate) response_timestamp: OffsetDateTime,
/// How many bytes from the last input chunk have we consumed
/// This is mostly used from callbacks, where the caller
/// wants to know how far into the last chunk the parser is.
pub(crate) response_bytes_consumed: usize,
/// Used to buffer a line of outbound data when buffering cannot be avoided.
pub(crate) response_buf: Bstr,
/// Stores the current value of a folded response header. Such headers span
/// multiple lines, and are processed only when all data is available.
pub(crate) response_header: Option<Bstr>,
/// The length of the current response body as presented in the
/// Content-Length response header.
pub(crate) response_content_length: Option<u64>,
/// The remaining length of the current response body, if known. Set to None otherwise.
pub(crate) response_body_data_left: Option<u64>,
/// Holds the amount of data that needs to be read from the
/// current response data chunk. Only used with chunked response bodies.
pub(crate) response_chunked_length: Option<u64>,
/// Current response parser state.
pub(crate) response_state: State,
/// Previous response parser state.
pub(crate) response_state_previous: State,
/// The hook that should be receiving raw connection data.
pub(crate) response_data_receiver_hook: Option<DataHook>,
/// Transactions processed by this parser
transactions: Transactions,
}
impl std::fmt::Debug for ConnectionParser {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
f.debug_struct("ConnectionParser")
.field("request_status", &self.request_status)
.field("response_status", &self.response_status)
.field("request_index", &self.request_index())
.field("response_index", &self.response_index())
.finish()
}
}
impl ConnectionParser {
/// Creates a new ConnectionParser with a preconfigured `Config` struct.
pub(crate) fn new(cfg: &'static Config) -> Self {
let conn = Connection::default();
let logger = Logger::new(conn.get_sender());
Self {
logger: logger.clone(),
cfg,
conn,
user_data: None,
request_status: HtpStreamState::NEW,
response_status: HtpStreamState::NEW,
response_data_other_at_tx_end: false,
request_timestamp: OffsetDateTime::from(SystemTime::now()),
request_bytes_consumed: 0,
request_chunk_count: 0,
request_chunk_request_index: 0,
request_buf: Bstr::new(),
request_header: None,
request_content_length: None,
request_body_data_left: None,
request_chunked_length: None,
request_state: State::Idle,
request_state_previous: State::None,
request_data_receiver_hook: None,
response_timestamp: OffsetDateTime::from(SystemTime::now()),
response_bytes_consumed: 0,
response_buf: Bstr::new(),
response_header: None,
response_content_length: None,
response_body_data_left: None,
response_chunked_length: None,
response_state: State::Idle,
response_state_previous: State::None,
response_data_receiver_hook: None,
transactions: Transactions::new(cfg, &logger),
}
}
/// Get the current request transaction
pub(crate) fn request(&mut self) -> Option<&Transaction> {
self.transactions.request()
}
/// Get the current request transaction
pub(crate) fn request_mut(&mut self) -> Option<&mut Transaction> {
self.transactions.request_mut()
}
/// Get the current response transaction
pub(crate) fn response(&mut self) -> Option<&Transaction> {
self.transactions.response()
}
/// Get the current response transaction
pub(crate) fn response_mut(&mut self) -> Option<&mut Transaction> {
self.transactions.response_mut()
}
/// Advance to the next request
/// Returns the next request transaction id
pub(crate) fn request_next(&mut self) -> usize {
// Detect pipelining.
if self.transactions.request_index() > self.transactions.response_index() {
self.conn.flags.set(ConnectionFlags::PIPELINED)
}
self.transactions.request_next()
}
/// Advance to the next response
/// Returns the next response transaction id
pub(crate) fn response_next(&mut self) -> usize {
self.transactions.response_next()
}
/// Get the index of the request transaction
pub(crate) fn request_index(&self) -> usize {
self.transactions.request_index()
}
/// Get the index of the response transaction
pub(crate) fn response_index(&self) -> usize {
self.transactions.response_index()
}
/// Get the number of transactions processed up to now
pub(crate) fn tx_size(&self) -> usize {
self.transactions.size()
}
/// Get a specific transaction
pub(crate) fn tx(&self, index: usize) -> Option<&Transaction> {
self.transactions.get(index)
}
/// Get a specific transaction
pub(crate) fn tx_mut(&mut self, index: usize) -> Option<&mut Transaction> {
self.transactions.get_mut(index)
}
/// Handle the current state to be processed.
pub(crate) fn handle_request_state(&mut self, data: &mut ParserData) -> Result<()> {
match self.request_state {
State::None => Err(HtpStatus::ERROR),
State::Idle => self.request_idle(data),
State::IgnoreDataAfterHTTP09 => self.request_ignore_data_after_http_0_9(data),
State::Line => self.request_line(data),
State::Protocol => self.request_protocol(data),
State::Headers => self.request_headers(data),
State::ConnectWaitResponse => self.request_connect_wait_response(),
State::ConnectCheck => self.request_connect_check(),
State::ConnectProbeData => self.request_connect_probe_data(data),
State::BodyDetermine => self.request_body_determine(),
State::BodyChunkedData => self.request_body_chunked_data(data),
State::BodyChunkedLength => self.request_body_chunked_length(data),
State::BodyChunkedDataEnd => self.request_body_chunked_data_end(data),
State::BodyIdentity => self.request_body_identity(data),
State::Finalize => self.request_finalize(data),
// These are only used by response_state
_ => Err(HtpStatus::ERROR),
}
}
/// Handle the current state to be processed.
pub(crate) fn handle_response_state(&mut self, data: &mut ParserData) -> Result<()> {
match self.response_state {
State::None => Err(HtpStatus::ERROR),
State::Idle => self.response_idle(data),
State::Line => self.response_line(data),
State::Headers => self.response_headers(data),
State::BodyDetermine => self.response_body_determine(data),
State::BodyChunkedData => self.response_body_chunked_data(data),
State::BodyChunkedLength => self.response_body_chunked_length(data),
State::BodyChunkedDataEnd => self.response_body_chunked_data_end(data),
State::Finalize => self.response_finalize(data),
State::BodyIdentityStreamClose => self.response_body_identity_stream_close(data),
State::BodyIdentityCLKnown => self.response_body_identity_cl_known(data),
// These are only used by request_state
_ => Err(HtpStatus::ERROR),
}
}
/// Closes the connection associated with the supplied parser.
pub(crate) fn request_close(&mut self, timestamp: Option<OffsetDateTime>) {
// Update internal flags
if self.request_status != HtpStreamState::ERROR {
self.request_status = HtpStreamState::CLOSED
}
// Call the parsers one last time, which will allow them
// to process the events that depend on stream closure
self.request_data(ParserData::default(), timestamp);
}
/// Closes the connection associated with the supplied parser.
pub(crate) fn close(&mut self, timestamp: Option<OffsetDateTime>) {
// Close the underlying connection.
self.conn.close(timestamp);
// Update internal flags
if self.request_status != HtpStreamState::ERROR {
self.request_status = HtpStreamState::CLOSED
}
if self.response_status != HtpStreamState::ERROR {
self.response_status = HtpStreamState::CLOSED
}
// Call the parsers one last time, which will allow them
// to process the events that depend on stream closure
self.request_data(ParserData::default(), timestamp);
self.response_data(ParserData::default(), timestamp);
}
/// This function is most likely not used and/or not needed.
pub(crate) fn request_reset(&mut self) {
self.request_content_length = None;
self.request_body_data_left = None;
self.request_chunk_request_index = self.request_chunk_count;
}
/// Returns the number of bytes consumed from the current data chunks so far.
pub(crate) fn request_data_consumed(&self) -> usize {
self.request_bytes_consumed
}
/// Consume the given number of bytes from the ParserData and update
/// the internal counter for how many bytes consumed so far.
pub(crate) fn request_data_consume(&mut self, input: &ParserData, consumed: usize) {
input.consume(consumed);
self.request_bytes_consumed = input.consumed_len();
}
/// Unconsume the given number of bytes from the ParserData and update the
/// the internal counter for how many bytes are consumed.
/// If the requested number of bytes is larger than the number of bytes
/// already consumed then the parser will be unwound to the beginning.
pub(crate) fn request_data_unconsume(&mut self, input: &mut ParserData, unconsume: usize) {
input.unconsume(unconsume);
self.request_bytes_consumed = input.consumed_len();
}
/// Consume the given number of bytes from the ParserData and update
/// the internal counter for how many bytes consumed so far.
pub(crate) fn response_data_consume(&mut self, input: &ParserData, consumed: usize) {
input.consume(consumed);
self.response_bytes_consumed = input.consumed_len();
}
/// Unconsume the given number of bytes from the ParserData and update the
/// the internal counter for how many bytes are consumed.
/// If the requested number of bytes is larger than the number of bytes
/// already consumed then the parser will be unwound to the beginning.
pub(crate) fn response_data_unconsume(&mut self, input: &mut ParserData, unconsume: usize) {
input.unconsume(unconsume);
self.response_bytes_consumed = input.consumed_len();
}
/// Returns the number of bytes consumed from the most recent outbound data chunk. Normally, an invocation
/// of response_data() will consume all data from the supplied buffer, but there are circumstances
/// where only partial consumption is possible. In such cases DATA_OTHER will be returned.
/// Consumed bytes are no longer necessary, but the remainder of the buffer will be saved
/// for later.
pub(crate) fn response_data_consumed(&self) -> usize {
self.response_bytes_consumed
}
/// Opens connection.
pub(crate) fn open(
&mut self, client_addr: Option<IpAddr>, client_port: Option<u16>,
server_addr: Option<IpAddr>, server_port: Option<u16>, timestamp: Option<OffsetDateTime>,
) {
// Check connection parser state first.
if self.request_status != HtpStreamState::NEW || self.response_status != HtpStreamState::NEW
{
htp_error!(
self.logger,
HtpLogCode::CONNECTION_ALREADY_OPEN,
"Connection is already open"
);
return;
}
self.conn.open(
client_addr,
client_port,
server_addr,
server_port,
timestamp,
);
self.request_status = HtpStreamState::OPEN;
self.response_status = HtpStreamState::OPEN;
}
/// Set the user data.
pub(crate) fn set_user_data(&mut self, data: Box<dyn Any + 'static>) {
self.user_data = Some(data);
}
/// Get a reference to the user data.
pub(crate) fn user_data<T: 'static>(&self) -> Option<&T> {
self.user_data
.as_ref()
.and_then(|ud| ud.downcast_ref::<T>())
}
/// Initialize request parsing, change state to LINE,
/// and invoke all registered callbacks.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP if one of the
/// callbacks does not want to follow the transaction any more.
pub(crate) fn state_request_start(&mut self) -> Result<()> {
// Change state into request line parsing.
self.request_state = State::Line;
let req = self.request_mut();
if req.is_none() {
return Err(HtpStatus::ERROR);
}
req.unwrap().request_progress = HtpRequestProgress::LINE;
// Run hook REQUEST_START.
self.cfg
.hook_request_start
.clone()
.run_all(self, self.request_index())?;
Ok(())
}
/// Change transaction state to HEADERS and invoke all
/// registered callbacks.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP if one of the
/// callbacks does not want to follow the transaction any more.
pub(crate) fn state_request_headers(&mut self, input: &mut ParserData) -> Result<()> {
// Finalize sending raw header data
self.request_receiver_finalize_clear(input)?;
// If we're in HTP_REQ_HEADERS that means that this is the
// first time we're processing headers in a request. Otherwise,
// we're dealing with trailing headers.
let req = self.request();
if req.is_none() {
return Err(HtpStatus::ERROR);
}
let request_progress = req.unwrap().request_progress;
if request_progress > HtpRequestProgress::HEADERS {
// Request trailers.
// Run hook HTP_REQUEST_TRAILER.
self.cfg
.hook_request_trailer
.clone()
.run_all(self, self.request_index())?;
// Completed parsing this request; finalize it now.
self.request_state = State::Finalize;
} else if request_progress >= HtpRequestProgress::LINE {
// Request headers.
// Did this request arrive in multiple data chunks?
let req = self.transactions.request_mut().unwrap();
if self.request_chunk_count != self.request_chunk_request_index {
req.flags.set(HtpFlags::MULTI_PACKET_HEAD)
}
req.process_request_headers()?;
// Run hook REQUEST_HEADERS.
#[cfg(test)]
self.cfg
.hook_request_headers
.clone()
.run_all(self, self.request_index())?;
self.request_initialize_decompressors()?;
// We still proceed if the request is invalid.
self.request_state = State::ConnectCheck;
} else {
htp_warn!(
self.logger,
HtpLogCode::RESPONSE_BODY_INTERNAL_ERROR,
format!(
"[Internal Error] Invalid tx progress: {:?}",
request_progress
)
);
return Err(HtpStatus::ERROR);
}
Ok(())
}
/// Change transaction state to PROTOCOL and invoke all
/// registered callbacks.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP if one of the
/// callbacks does not want to follow the transaction any more.
pub(crate) fn state_request_line(&mut self) -> Result<()> {
let req = self.request_mut();
if req.is_none() {
return Err(HtpStatus::ERROR);
}
req.unwrap().parse_request_line()?;
// Run hook REQUEST_LINE.
self.cfg
.hook_request_line
.clone()
.run_all(self, self.request_index())?;
let logger = self.logger.clone();
let req = self.request_mut().unwrap();
if let Some(parsed_uri) = req.parsed_uri.as_mut() {
let (partial_normalized_uri, complete_normalized_uri) =
parsed_uri.generate_normalized_uri(Some(logger));
req.partial_normalized_uri = partial_normalized_uri;
req.complete_normalized_uri = complete_normalized_uri;
}
// Move on to the next phase.
self.request_state = State::Protocol;
Ok(())
}
/// Advance state after processing request headers.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP
/// if one of the callbacks does not want to follow the transaction any more.
pub(crate) fn state_request_complete(&mut self, input: &mut ParserData) -> Result<()> {
let req = self.request_mut();
if req.is_none() {
return Err(HtpStatus::ERROR);
}
let req = req.unwrap();
if req.request_progress != HtpRequestProgress::COMPLETE {
// Finalize request body.
if req.request_has_body() {
self.request_body_data(None)?;
}
self.request_mut().unwrap().request_progress = HtpRequestProgress::COMPLETE;
// Run hook REQUEST_COMPLETE.
self.cfg
.hook_request_complete
.clone()
.run_all(self, self.request_index())?;
// Clear request data
self.request_receiver_finalize_clear(input)?;
}
// Determine what happens next, and remove this transaction from the parser.
self.request_state = if self.request().unwrap().is_protocol_0_9 {
State::IgnoreDataAfterHTTP09
} else {
State::Idle
};
// Check if the entire transaction is complete.
self.finalize(self.request_index())?;
self.request_next();
Ok(())
}
/// Determine if the transaction is complete and run any hooks.
fn finalize(&mut self, _tx_index: usize) -> Result<()> {
#[cfg(test)]
if let Some(tx) = self.tx(_tx_index) {
if !tx.is_complete() {
return Ok(());
}
// Disconnect transaction from the parser.
// Run hook TRANSACTION_COMPLETE.
self.cfg
.hook_transaction_complete
.clone()
.run_all(self, _tx_index)?;
}
Ok(())
}
/// Advance state to LINE, or BODY if http version is 0.9.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP
/// if one of the callbacks does not want to follow the transaction any more.
pub(crate) fn state_response_start(&mut self) -> Result<()> {
// Change state into response line parsing, except if we're following
// a HTTP/0.9 request (no status line or response headers).
let tx = self.response_mut();
if tx.is_none() {
return Err(HtpStatus::ERROR);
}
let tx = tx.unwrap();
if tx.is_protocol_0_9 {
tx.response_transfer_coding = HtpTransferCoding::Identity;
tx.response_content_encoding_processing = HtpContentEncoding::None;
tx.response_progress = HtpResponseProgress::BODY;
self.response_state = State::BodyIdentityStreamClose;
self.response_body_data_left = None
} else {
tx.response_progress = HtpResponseProgress::LINE;
self.response_state = State::Line
}
// Run hook RESPONSE_START.
self.cfg
.hook_response_start
.clone()
.run_all(self, self.response_index())?;
// If at this point we have no method and no uri and our status
// is still REQ_LINE, we likely have timed out request
// or a overly long request
let tx = self.response_mut().unwrap();
if tx.request_method.is_none()
&& tx.request_uri.is_none()
&& self.request_state == State::Line
{
htp_warn!(
self.logger,
HtpLogCode::REQUEST_LINE_INCOMPLETE,
"Request line incomplete"
);
}
Ok(())
}
/// Advance state after processing response headers.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP
/// if one of the callbacks does not want to follow the transaction any more.
pub(crate) fn state_response_headers(&mut self, input: &mut ParserData) -> Result<()> {
// Finalize sending raw header data.
self.response_receiver_finalize_clear(input)?;
// Run hook RESPONSE_HEADERS.
#[cfg(test)]
self.cfg
.hook_response_headers
.clone()
.run_all(self, self.response_index())?;
self.response_initialize_decompressors()
}
/// Change transaction state to RESPONSE_LINE and invoke registered callbacks.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP
/// if one of the callbacks does not want to follow the transaction any more.
pub(crate) fn state_response_line(&mut self) -> Result<()> {
// Is the response line valid?
let tx = self.response_mut();
if tx.is_none() {
return Err(HtpStatus::ERROR);
}
let tx = tx.unwrap();
tx.validate_response_line();
#[cfg(test)]
let index = tx.index;
// Run hook HTP_RESPONSE_LINE
#[cfg(test)]
return self.cfg.hook_response_line.clone().run_all(self, index);
#[cfg(not(test))]
return Ok(());
}
/// Change transaction state to COMPLETE and invoke registered callbacks.
///
/// Returns HtpStatus::OK on success; HtpStatus::ERROR on error, HtpStatus::STOP
/// if one of the callbacks does not want to follow the transaction any more.
pub(crate) fn state_response_complete(&mut self, input: &mut ParserData) -> Result<()> {
let response_index = self.response_index();
let tx = self.response_mut();
if tx.is_none() {
return Err(HtpStatus::ERROR);
}
let tx = tx.unwrap();
if tx.response_progress != HtpResponseProgress::COMPLETE {
tx.response_progress = HtpResponseProgress::COMPLETE;
// Run the last RESPONSE_BODY_DATA HOOK, but only if there was a response body present.
if tx.response_transfer_coding != HtpTransferCoding::NoBody {
let _ = self.response_body_data(None);
}
// Run hook RESPONSE_COMPLETE.
self.cfg
.hook_response_complete
.clone()
.run_all(self, response_index)?;
// Clear the data receivers hook if any
self.response_receiver_finalize_clear(input)?;
}
// Check if we want to signal the caller to send request data
self.request_parser_check_waiting()?;
// Otherwise finalize the transaction
self.finalize(response_index)?;
self.response_next();
self.response_state = State::Idle;
Ok(())
}
/// Check if we had previously signalled the caller to give us response
/// data, and now we are ready to receive it
fn request_parser_check_waiting(&mut self) -> Result<()> {
// Check if the inbound parser is waiting on us. If it is, that means that
// there might be request data that the inbound parser hasn't consumed yet.
// If we don't stop parsing we might encounter a response without a request,
// which is why we want to return straight away before processing any data.
//
// This situation will occur any time the parser needs to see the server
// respond to a particular situation before it can decide how to proceed. For
// example, when a CONNECT is sent, different paths are used when it is accepted
// and when it is not accepted.
//
// It is not enough to check only in_status here. Because of pipelining, it's possible
// that many inbound transactions have been processed, and that the parser is
// waiting on a response that we have not seen yet.
if self.response_status == HtpStreamState::DATA_OTHER
&& self.response_index() == self.request_index()
{
return Err(HtpStatus::DATA_OTHER);
}
// Do we have a signal to yield to inbound processing at
// the end of the next transaction?
if self.response_data_other_at_tx_end {
// We do. Let's yield then.
self.response_data_other_at_tx_end = false;
return Err(HtpStatus::DATA_OTHER);
}
Ok(())
}
/// Remove the given transaction from the parser
pub(crate) fn remove_tx(&mut self, tx_id: usize) {
self.transactions.remove(tx_id);
}
}
rust/htp/src/decompressors.rs
+1032
View File
File diff suppressed because it is too large Load Diff
rust/htp/src/error.rs
+47
View File
@@ -0,0 +1,47 @@
use crate::HtpStatus;
use nom::error::ErrorKind as NomErrorKind;
/// Helper for nom's default error type
pub(crate) type NomError<I> = nom::error::Error<I>;
/// Alias for libhtp Result type. Result types are classified by `HtpStatus`.
pub(crate) type Result<T> = std::result::Result<T, HtpStatus>;
impl<T> From<Result<T>> for HtpStatus {
/// Returns HtpStatus from result.
fn from(res: Result<T>) -> HtpStatus {
match res {
Ok(_) => HtpStatus::OK,
Err(e) => e,
}
}
}
impl From<HtpStatus> for Result<()> {
/// Returns Result from `HtpStatus`
fn from(status: HtpStatus) -> Result<()> {
if status == HtpStatus::OK {
Ok(())
} else {
Err(status)
}
}
}
impl From<std::io::Error> for HtpStatus {
fn from(_: std::io::Error) -> Self {
HtpStatus::ERROR
}
}
impl<I: std::fmt::Debug> From<nom::Err<NomError<I>>> for HtpStatus {
fn from(_: nom::Err<NomError<I>>) -> Self {
HtpStatus::ERROR
}
}
impl From<NomErrorKind> for HtpStatus {
fn from(_: NomErrorKind) -> Self {
HtpStatus::ERROR
}
}
rust/htp/src/headers.rs
+968
View File
@@ -0,0 +1,968 @@
use crate::util::{is_token, trimmed, FlagOperations};
use nom::{
branch::alt,
bytes::complete::tag as complete_tag,
bytes::streaming::{tag, take_till, take_while, take_while1},
character::{is_space, streaming::space0},
combinator::{complete, map, not, opt, peek},
sequence::tuple,
Err::Incomplete,
IResult, Needed,
};
/// Helper for Parsed bytes and corresponding HeaderFlags
pub(crate) type ParsedBytes<'a> = (&'a [u8], u64);
// Helper for Parsed Headers and corresonding termination
pub(crate) type ParsedHeaders = (Vec<Header>, bool);
// Helper for matched eol+ folding bytes + flags
pub(crate) type FoldingBytes<'a> = (&'a [u8], &'a [u8], u64);
// Helper for folding or terminator bytes
pub(crate) type FoldingOrTerminator<'a> = (ParsedBytes<'a>, Option<&'a [u8]>);
// Helper for value bytes and the value terminator
pub(crate) type ValueBytes<'a> = (&'a [u8], FoldingOrTerminator<'a>);
#[repr(C)]
#[derive(Debug, PartialEq, Eq)]
pub(crate) struct HeaderFlags;
impl HeaderFlags {
pub(crate) const FOLDING: u64 = 0x0001;
pub(crate) const FOLDING_SPECIAL_CASE: u64 = (0x0002 | Self::FOLDING);
pub(crate) const NAME_EMPTY: u64 = 0x0004;
pub(crate) const VALUE_EMPTY: u64 = 0x0008;
pub(crate) const NAME_NON_TOKEN_CHARS: u64 = 0x0010;
pub(crate) const FIELD_REPEATED: u64 = 0x0020;
pub(crate) const NAME_TRAILING_WHITESPACE: u64 = 0x0040;
pub(crate) const NAME_LEADING_WHITESPACE: u64 = 0x0080;
pub(crate) const NULL_TERMINATED: u64 = 0x0100;
pub(crate) const MISSING_COLON: u64 = (0x0200 | Self::NAME_EMPTY);
pub(crate) const DEFORMED_EOL: u64 = 0x0400;
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Name {
pub(crate) name: Vec<u8>,
pub(crate) flags: u64,
}
impl Name {
pub(crate) fn new(name: &[u8], flags: u64) -> Self {
Self {
name: trimmed(name).to_vec(),
flags,
}
}
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Value {
pub(crate) value: Vec<u8>,
pub(crate) flags: u64,
}
impl Value {
pub(crate) fn new(value: &[u8], flags: u64) -> Self {
Self {
value: trimmed(value).to_vec(),
flags,
}
}
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Header {
pub(crate) name: Name,
pub(crate) value: Value,
}
impl Header {
pub(crate) fn new(name: Name, value: Value) -> Self {
Self { name, value }
}
pub(crate) fn new_with_flags(
name_bytes: &[u8], name_flags: u64, value_bytes: &[u8], value_flags: u64,
) -> Self {
Self::new(
Name::new(name_bytes, name_flags),
Value::new(value_bytes, value_flags),
)
}
}
/// Enumerates possible parser types
#[derive(PartialEq, Eq, Copy, Clone, Debug)]
pub(crate) enum Side {
/// Request Parser: null terminates
Request,
/// Response Parser: accepts CR as a line ending
Response,
}
pub(crate) struct Parser {
side: Side,
complete: bool,
}
impl Parser {
pub(crate) fn new(side: Side) -> Self {
Self {
side,
complete: false,
}
}
/// Sets the parser complete state.
///
/// If set to true, parser operates under the assumption that no more data is incoming
pub(crate) fn set_complete(&mut self, complete: bool) {
self.complete = complete;
}
/// Returns true if c is a line feed character
fn is_eol(&self) -> impl Fn(u8) -> bool + '_ {
move |c| c == b'\n' || (self.side == Side::Response && c == b'\r')
}
/// Parse one complete end of line character or character set
fn complete_eol_regular(&self) -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> + '_ {
move |input| {
if self.side == Side::Response {
alt((
complete_tag("\r\n"),
complete_tag("\n\r"),
complete_tag("\n"),
complete_tag("\r"),
))(input)
} else {
alt((complete_tag("\r\n"), complete_tag("\n")))(input)
}
}
}
/// Parse one complete deformed end of line character set
fn complete_eol_deformed(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedBytes> + '_ {
move |input| {
if self.side == Side::Response {
alt((
map(
tuple((
complete_tag("\n\r\r\n"),
peek(alt((complete_tag("\n"), complete_tag("\r\n")))),
)),
|(eol, _)| (eol, HeaderFlags::DEFORMED_EOL),
),
map(
tuple((
complete_tag("\r\n\r"),
take_while1(|c| c == b'\r' || c == b' ' || c == b'\t'),
opt(complete_tag("\n")),
not(alt((complete_tag("\n"), complete_tag("\r\n")))),
)),
|(eol1, eol2, eol3, _): (&[u8], &[u8], Option<&[u8]>, _)| {
(
&input[..(eol1.len() + eol2.len() + eol3.unwrap_or(b"").len())],
HeaderFlags::DEFORMED_EOL,
)
},
),
))(input)
} else {
map(
alt((
tuple((
complete_tag("\n\r\r\n"),
peek(alt((complete_tag("\n"), complete_tag("\r\n")))),
)),
tuple((complete_tag("\n\r"), peek(complete_tag("\r\n")))),
)),
|(eol, _)| (eol, HeaderFlags::DEFORMED_EOL),
)(input)
}
}
}
/// Parse one complete end of line character or character set
fn complete_eol(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedBytes> + '_ {
move |input| {
alt((
self.complete_eol_deformed(),
map(self.complete_eol_regular(), |eol| (eol, 0)),
))(input)
}
}
/// Parse one header end of line, and guarantee that it is not folding
fn eol(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedBytes> + '_ {
move |input| {
map(
tuple((self.complete_eol(), not(folding_lws))),
|(end, _)| end,
)(input)
}
}
/// Parse one null byte or one end of line, and guarantee that it is not folding
fn null_or_eol(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedBytes> + '_ {
move |input| alt((null, self.eol()))(input)
}
/// Parse one null byte or complete end of line
fn complete_null_or_eol(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedBytes> + '_ {
move |input| alt((null, self.complete_eol()))(input)
}
/// Parse header folding bytes (eol + whitespace or eol + special cases)
fn folding(&self) -> impl Fn(&[u8]) -> IResult<&[u8], FoldingBytes> + '_ {
move |input| {
if self.side == Side::Response {
map(
tuple((
map(self.complete_eol_regular(), |eol| (eol, 0)),
folding_lws,
)),
|((eol, flags), (lws, other_flags))| (eol, lws, flags | other_flags),
)(input)
} else {
map(
tuple((self.complete_eol(), folding_lws)),
|((eol, flags), (lws, other_flags))| (eol, lws, flags | other_flags),
)(input)
}
}
}
/// Parse complete folding bytes or a value terminator (eol or null)
fn complete_folding_or_terminator(
&self,
) -> impl Fn(&[u8]) -> IResult<&[u8], FoldingOrTerminator> + '_ {
move |input| {
alt((
complete(map(self.folding(), |(end, fold, flags)| {
((end, flags), Some(fold))
})),
map(self.complete_null_or_eol(), |end| (end, None)),
))(input)
}
}
/// Parse complete folding bytes or a value terminator (eol or null)
fn streaming_folding_or_terminator(
&self,
) -> impl Fn(&[u8]) -> IResult<&[u8], FoldingOrTerminator> + '_ {
move |input| {
alt((
map(self.folding(), |(end, fold, flags)| {
((end, flags), Some(fold))
}),
map(self.null_or_eol(), |end| (end, None)),
))(input)
}
}
/// Parse folding bytes or a value terminator (eol or null)
fn folding_or_terminator(&self) -> impl Fn(&[u8]) -> IResult<&[u8], FoldingOrTerminator> + '_ {
move |input| {
if self.complete {
self.complete_folding_or_terminator()(input)
} else {
self.streaming_folding_or_terminator()(input)
}
}
}
/// Parse a header value.
/// Returns the bytes and the value terminator; null, eol or folding
/// eg. (bytes, (eol_bytes, Option<fold_bytes>))
fn value_bytes(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ValueBytes> + '_ {
move |input| {
let (mut remaining, mut value) = take_till(self.is_eol())(input)?;
if value.last() == Some(&b'\r') {
value = &value[..value.len() - 1];
remaining = &input[value.len()..];
}
let (remaining, result) = self.folding_or_terminator()(remaining)?;
Ok((remaining, (value, result)))
}
}
/// Parse a complete header value, including any folded headers
fn value(&self) -> impl Fn(&[u8]) -> IResult<&[u8], Value> + '_ {
move |input| {
let (mut rest, (val_bytes, ((_eol, mut flags), fold))) = self.value_bytes()(input)?;
let mut value = val_bytes.to_vec();
if let Some(fold) = fold {
let mut i = rest;
let mut ofold = fold;
loop {
if self.side == Side::Response {
// Peek ahead for ambiguous name with lws vs. value with folding
match tuple((token_chars, separator_regular))(i) {
Ok((_, ((_, tokens, _), (_, _)))) if !tokens.is_empty() => {
flags.unset(HeaderFlags::FOLDING_SPECIAL_CASE);
if value.is_empty() {
flags.set(HeaderFlags::VALUE_EMPTY);
}
// i is now the latest rest
return Ok((i, Value::new(&value, flags)));
}
Err(Incomplete(_)) => {
return Err(Incomplete(Needed::new(1)));
}
_ => {}
}
}
let (rest2, (val_bytes, ((eol, other_flags), fold))) = self.value_bytes()(i)?;
i = rest2;
flags.set(other_flags);
//If the value is empty, the value started with a fold and we don't want to push back a space
if !value.is_empty() {
if !ofold.is_empty() {
value.push(ofold[0]);
} else {
value.push(b' ');
}
}
if !val_bytes.is_empty() || eol.len() > 1 {
// we keep empty folding as a future new eol
rest = rest2;
value.extend(val_bytes);
} else if val_bytes.is_empty()
&& eol.len() == 1
&& !rest2.is_empty()
&& rest2[0] == b'\n'
{
// eol empty fold double eol is enfo of headers
rest = rest2;
}
if let Some(fold) = fold {
ofold = fold;
} else {
return Ok((rest, Value::new(&value, flags)));
}
}
} else {
if value.is_empty() {
flags.set(HeaderFlags::VALUE_EMPTY);
}
Ok((rest, Value::new(&value, flags)))
}
}
}
/// Parse one header name
fn name(&self) -> impl Fn(&[u8]) -> IResult<&[u8], Name> + '_ {
move |input| {
let mut terminated = 0;
let mut offset = 0;
for (i, c) in input.iter().enumerate() {
if terminated == 0 {
if *c == b':' {
offset = i;
break;
} else if *c == b'\n' || (self.side == Side::Response && *c == b'\r') {
terminated = *c;
}
} else if *c == b' ' {
terminated = 0;
} else if *c == b'\n' && terminated == b'\r' {
terminated = *c;
} else {
offset = i - 1;
break;
}
}
let (name, rem) = input.split_at(offset);
let mut flags = 0;
if !name.is_empty() {
if is_space(name[0]) {
flags.set(HeaderFlags::NAME_LEADING_WHITESPACE)
}
if let Some(end) = name.last() {
if is_space(*end) {
flags.set(HeaderFlags::NAME_TRAILING_WHITESPACE);
}
}
if let Ok((rem, _)) = token_chars(name) {
if !rem.is_empty() {
flags.set(HeaderFlags::NAME_NON_TOKEN_CHARS);
}
}
} else {
flags.set(HeaderFlags::NAME_EMPTY)
}
Ok((rem, Name::new(name, flags)))
}
}
/// Parse a separator between header name and value
fn separator(&self) -> impl Fn(&[u8]) -> IResult<&[u8], u64> + '_ {
move |input| map(separator_regular, |_| 0)(input)
}
/// Parse data before an eol with no colon as an empty name with the data as the value
fn header_sans_colon(&self) -> impl Fn(&[u8]) -> IResult<&[u8], Header> + '_ {
move |input| {
let (remaining, (_, value)) = tuple((not(complete_tag("\r\n")), self.value()))(input)?;
let flags = value.flags | HeaderFlags::MISSING_COLON;
Ok((
remaining,
Header::new_with_flags(b"", flags, &value.value, flags),
))
}
}
/// Parse a header name separator value
fn header_with_colon(&self) -> impl Fn(&[u8]) -> IResult<&[u8], Header> + '_ {
move |input| {
map(
tuple((self.name(), self.separator(), self.value())),
|(mut name, flag, mut value)| {
name.flags |= flag;
value.flags |= flag;
Header::new(name, value)
},
)(input)
}
}
/// Parses a header name and value with, or without a colon separator
fn header(&self) -> impl Fn(&[u8]) -> IResult<&[u8], Header> + '_ {
move |input| alt((complete(self.header_with_colon()), self.header_sans_colon()))(input)
}
/// Parse multiple headers and indicate if end of headers or null was found
pub(crate) fn headers(&self) -> impl Fn(&[u8]) -> IResult<&[u8], ParsedHeaders> + '_ {
move |input| {
let mut out = Vec::with_capacity(16);
let mut i = input;
loop {
match self.header()(i) {
Ok((rest, head)) => {
i = rest;
let is_null_terminated =
head.value.flags.is_set(HeaderFlags::NULL_TERMINATED);
out.push(head);
if is_null_terminated {
return Ok((rest, (out, true)));
}
if let Ok((rest2, _eoh)) = self.complete_eol_regular()(rest) {
return Ok((rest2, (out, true)));
}
}
Err(Incomplete(x)) => {
if out.is_empty() {
return Err(Incomplete(x));
}
return Ok((i, (out, false)));
}
Err(e) => {
if out.is_empty() {
if let Ok((rest2, _eoh)) = self.complete_eol()(i) {
return Ok((rest2, (out, true)));
}
}
return Err(e);
}
}
}
}
}
}
/// Parse one null character and return it and the NULL_TERMINATED flag
fn null(input: &[u8]) -> IResult<&[u8], ParsedBytes> {
map(complete_tag("\0"), |null| {
(null, HeaderFlags::NULL_TERMINATED)
})(input)
}
/// Extracts folding lws (whitespace only)
fn folding_lws(input: &[u8]) -> IResult<&[u8], ParsedBytes> {
map(alt((tag(" "), tag("\t"), tag("\0"))), |fold| {
(fold, HeaderFlags::FOLDING)
})(input)
}
/// Parse a regular separator (colon followed by optional spaces) between header name and value
fn separator_regular(input: &[u8]) -> IResult<&[u8], (&[u8], &[u8])> {
tuple((complete_tag(":"), space0))(input)
}
type leading_token_trailing<'a> = (&'a [u8], &'a [u8], &'a [u8]);
/// Parse token characters with leading and trailing whitespace
fn token_chars(input: &[u8]) -> IResult<&[u8], leading_token_trailing> {
tuple((space0, take_while(is_token), space0))(input)
}
#[cfg(test)]
mod test {
use super::*;
use crate::error::NomError;
use nom::{
error::ErrorKind::{Not, Tag},
Err::{Error, Incomplete},
Needed,
};
use rstest::rstest;
macro_rules! b {
($b: literal) => {
$b.as_bytes()
};
}
// Helper for matched leading whitespace, byes, and trailing whitespace
pub(crate) type SurroundedBytes<'a> = (&'a [u8], &'a [u8], &'a [u8]);
#[rstest]
#[case::null_does_not_terminate(b"k1:v1\r\nk2:v2 before\0v2 after\r\n\r\n",Ok((b!(""), (vec![Header::new_with_flags(b"k1", 0, b"v1", 0), Header::new_with_flags(b"k2", 0, b"v2 before\0v2 after", 0)], true))), None)]
#[case::flags(b"k1:v1\r\n:v2\r\n v2+\r\nk3: v3\r\nk4 v4\r\nk\r5:v\r5\n\rmore\r\n\r\n", Ok((b!(""), (
vec![
Header::new_with_flags(b"k1", 0, b"v1", 0),
Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"v2 v2+", HeaderFlags::FOLDING),
Header::new_with_flags(b"k3", 0, b"v3", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"k4 v4", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"k\r5", HeaderFlags::NAME_NON_TOKEN_CHARS, b"v\r5", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"more", HeaderFlags::MISSING_COLON),
], true))), Some(Ok((b!(""), (
vec![
Header::new_with_flags(b"k1", 0, b"v1", 0),
Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"v2 v2+", HeaderFlags::FOLDING),
Header::new_with_flags(b"k3", 0, b"v3", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"k4 v4", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"k", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"5", 0, b"v", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"5", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"more", HeaderFlags::MISSING_COLON),
], true)))))]
#[case::incomplete_eoh(b"k1:v1\r\nk2:v2\r", Ok((b!("k2:v2\r"), (vec![Header::new_with_flags(b"k1", 0, b"v1", 0)], false))), None)]
#[case::incomplete_eoh_null(b"k1:v1\nk2:v2\0v2\r\nk3:v3\r", Ok((b!("k3:v3\r"), (vec![Header::new_with_flags(b"k1", 0, b"v1", 0), Header::new_with_flags(b"k2", 0, b"v2\0v2", 0)], false))), None)]
fn test_headers(
#[case] input: &[u8], #[case] expected: IResult<&[u8], ParsedHeaders>,
#[case] diff_res_expected: Option<IResult<&[u8], ParsedHeaders>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.headers()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.headers()(input), res_expected);
} else {
assert_eq!(res_parser.headers()(input), expected);
}
}
#[rstest]
#[case::only_lf_eoh(
b"Name1: Value1\nName2:Value2\nName3: Val\n ue3\nName4: Value4\n Value4.1\n Value4.2\n\n",
None
)]
#[case::only_crlf_eoh(b"Name1: Value1\r\nName2:Value2\r\nName3: Val\r\n ue3\r\nName4: Value4\r\n Value4.1\r\n Value4.2\r\n\r\n", None)]
#[case::crlf_lf_eoh(b"Name1: Value1\r\nName2:Value2\nName3: Val\r\n ue3\r\nName4: Value4\r\n Value4.1\n Value4.2\r\n\n", None)]
#[case::only_cr(b"Name1: Value1\rName2:Value2\rName3: Val\r\n ue3\rName4: Value4\r\n Value4.1\r\n Value4.2\r\r\n", Some(Err(Incomplete(Needed::new(1)))))]
#[case::cr_lf_crlf_eoh(b"Name1: Value1\rName2:Value2\rName3: Val\r\n ue3\r\nName4: Value4\r\n Value4.1\n Value4.2\r\n\n", Some(Ok((b!(""),
(
vec![
Header::new_with_flags(b"Name1", 0, b"Value1\rName2:Value2\rName3: Val ue3", HeaderFlags::FOLDING),
Header::new_with_flags(b"Name4", 0, b"Value4 Value4.1 Value4.2", HeaderFlags::FOLDING)
],
true
)))))]
#[case::crlf_lfcr_lf(b"Name1: Value1\r\nName2:Value2\nName3: Val\n\r ue3\n\rName4: Value4\r\n Value4.1\n Value4.2\r\n\n", Some(Ok((b!(""),
(
vec![
Header::new_with_flags(b"Name1", 0, b"Value1", 0),
Header::new_with_flags(b"Name2", 0, b"Value2", 0),
Header::new_with_flags(b"Name3", 0, b"Val", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"ue3", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"Name4", HeaderFlags::NAME_NON_TOKEN_CHARS, b"Value4 Value4.1 Value4.2", HeaderFlags::FOLDING),
],
true
)))))]
#[case::lfcr_eoh(b"Name1: Value1\n\rName2:Value2\n\rName3: Val\n\r ue3\n\rName4: Value4\n\r Value4.1\n\r Value4.2\n\r\n\r", Some(Ok((b!("\r"),
(
vec![
Header::new_with_flags(b"Name1", 0, b"Value1", 0),
Header::new_with_flags(b"Name2", HeaderFlags::NAME_NON_TOKEN_CHARS, b"Value2", 0),
Header::new_with_flags(b"Name3", HeaderFlags::NAME_NON_TOKEN_CHARS, b"Val", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"ue3", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"Name4", HeaderFlags::NAME_NON_TOKEN_CHARS, b"Value4", 0),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"Value4.1", HeaderFlags::MISSING_COLON),
Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"Value4.2", HeaderFlags::MISSING_COLON),
],
true
)))))]
fn test_headers_eoh(
#[case] input: &[u8], #[case] diff_req_expected: Option<IResult<&[u8], ParsedHeaders>>,
) {
let expected = Ok((
b!(""),
(
vec![
Header::new_with_flags(b"Name1", 0, b"Value1", 0),
Header::new_with_flags(b"Name2", 0, b"Value2", 0),
Header::new_with_flags(b"Name3", 0, b"Val ue3", HeaderFlags::FOLDING),
Header::new_with_flags(
b"Name4",
0,
b"Value4 Value4.1 Value4.2",
HeaderFlags::FOLDING,
),
],
true,
),
));
let req_parser = Parser::new(Side::Request);
let res_parser = Parser::new(Side::Response);
if let Some(req_expected) = diff_req_expected {
assert_eq!(req_parser.headers()(input), req_expected);
} else {
assert_eq!(req_parser.headers()(input), expected);
}
assert_eq!(res_parser.headers()(input), expected);
}
#[rstest]
#[case::incomplete(b"K V", Err(Incomplete(Needed::new(1))), None)]
#[case::contains_colon_1(b"K:V\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::contains_colon_2(b"K:V\r\nK2: V2", Ok((b!("K2: V2"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K:V", HeaderFlags::MISSING_COLON))), None)]
#[case::empty_name_value(b"\r\n", Err(Error(NomError::new(b!("\r\n"), Not))), None)]
#[case::contains_null(b"K V\0alue\r\nk", Ok((b!("k"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K V\0alue", HeaderFlags::MISSING_COLON))), None)]
#[case::folding(b"K V\ralue\r\nk", Ok((b!("k"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K V\ralue", HeaderFlags::MISSING_COLON))), Some(Ok((b!("alue\r\nk"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K V", HeaderFlags::MISSING_COLON)))))]
#[case::crlf(b"K V\r\nk1:v1\r\n", Ok((b!("k1:v1\r\n"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K V", HeaderFlags::MISSING_COLON))), None)]
#[case::lf(b"K V\nk1:v1\r\n", Ok((b!("k1:v1\r\n"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K V", HeaderFlags::MISSING_COLON))), None)]
fn test_header_sans_colon(
#[case] input: &[u8], #[case] expected: IResult<&[u8], Header>,
#[case] response_parser_expected: Option<IResult<&[u8], Header>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.header_sans_colon()(input), expected);
let res_parser = Parser::new(Side::Response);
let res_expected = if let Some(response_expected) = response_parser_expected {
response_expected
} else {
expected
};
assert_eq!(res_parser.header_sans_colon()(input), res_expected);
}
#[rstest]
#[case::incomplete(b"K: V", Err(Incomplete(Needed::new(1))))]
#[case::contains_colon(b"K: V\r\n", Err(Incomplete(Needed::new(1))))]
#[case::missing_colon(b"K V\nK:V\r\n", Err(Error(NomError::new(b!("\nK:V\r\n"), Tag))))]
#[case::contains_null(b":\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"", HeaderFlags::VALUE_EMPTY))))]
#[case::folding(b"K:\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"", HeaderFlags::VALUE_EMPTY))))]
#[case::crlf(b":V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"V", 0))))]
#[case::lf_1(b"K:folded\r\n\rV\r\n\r\n", Ok((b!("\rV\r\n\r\n"), Header::new_with_flags(b"K", 0, b"folded", 0))))]
#[case::lf_2(b"K: V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V", 0))))]
#[case::lf_3(b"K: V before\0 V after\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V before\0 V after", 0))))]
#[case::lf_4(b"K: V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V", 0))))]
#[case::lf_5(b"K: V before\0 V after\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V before\0 V after", 0))))]
#[case::lf_6(b"K: V\r\n a\r\n l\r\n u\r\n\te\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V a l u\te", HeaderFlags::FOLDING))))]
fn test_header_with_colon(#[case] input: &[u8], #[case] expected: IResult<&[u8], Header>) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.header_with_colon()(input), expected);
let res_parser = Parser::new(Side::Response);
assert_eq!(res_parser.header_with_colon()(input), expected);
}
#[rstest]
#[case::incomplete(b"K: V", Err(Incomplete(Needed::new(1))), None)]
#[case::contains_colon(b"K: V\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::missing_colon_1(b"K V\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::missing_colon_2(b"K1 V1\r\nK2:V2\n\r\n", Ok((b!("K2:V2\n\r\n"), Header::new_with_flags(b"", HeaderFlags::MISSING_COLON, b"K1 V1", HeaderFlags::MISSING_COLON))), None)]
#[case::empty_name_value(b"K1:V1\nK2:V2\n\r\n", Ok((b!("K2:V2\n\r\n"), Header::new_with_flags(b"K1", 0, b"V1", 0))), None)]
#[case::contains_null(b":\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"", HeaderFlags::VALUE_EMPTY))), None)]
#[case::folding(b"K:\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"", HeaderFlags::VALUE_EMPTY))), None)]
#[case::empty_name(b":V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"", HeaderFlags::NAME_EMPTY, b"V", 0))), None)]
#[case::special_folding(b"K:folded\r\n\rV\r\n\r\n", Ok((b!("\rV\r\n\r\n"), Header::new_with_flags(b"K", 0, b"folded", 0))), None)]
#[case::regular_eoh(b"K: V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V", 0))), None)]
#[case::folding(b"K: V\n a\r\n l\n u\r\n\te\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V a l u\te", HeaderFlags::FOLDING))), None)]
#[case::cr_in_name(b"Host:www.google.com\rName: Value\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"Host", 0, b"www.google.com\rName: Value", 0))), Some(Ok((b!("Name: Value\r\n\r\n"), Header::new_with_flags(b"Host", 0, b"www.google.com", 0)))))]
#[case::null_in_value(b"K: V before\0 V after\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V before\0 V after", 0))), None)]
#[case::folding(b"K: V\r a\r\n l\n u\r\n\te\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V\r a l u\te", HeaderFlags::FOLDING))), Some(Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"V a l u\te", HeaderFlags::FOLDING)))))]
#[case::deformed_folding_1(b"K:deformed folded\n\r V\n\r\r\n\n", Ok((b!("\r V\n\r\r\n\n"), Header::new_with_flags(b"K", 0, b"deformed folded", 0))), Some(Ok((b!("\n"), Header::new_with_flags(b"K", 0, b"deformed folded V", HeaderFlags::FOLDING | HeaderFlags::DEFORMED_EOL)))))]
#[case::deformed_folding_2(b"K:deformed folded\n\r V\r\n\r\n", Ok(( b!("\r V\r\n\r\n"), Header::new_with_flags(b"K", 0, b"deformed folded", 0))), Some(Ok((b!("\r\n"), Header::new_with_flags(b"K", 0, b"deformed folded V", HeaderFlags::FOLDING)))))]
#[case::deformed_folding_3(b"K:deformed folded\n\r\r V\r\n\r\n", Ok(( b!("\r\r V\r\n\r\n"), Header::new_with_flags(b"K", 0, b"deformed folded", 0))), Some(Ok((b!("\r V\r\n\r\n"), Header::new_with_flags(b"K", 0, b"deformed folded", 0)))))]
#[case::non_token_trailing_ws(b"K\r \r :\r V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K\r \r ", HeaderFlags::NAME_NON_TOKEN_CHARS | HeaderFlags::NAME_TRAILING_WHITESPACE, b"\r V", 0))), Some(Ok((b!("\r\n"), Header::new_with_flags(b"K", HeaderFlags::NAME_NON_TOKEN_CHARS | HeaderFlags::NAME_TRAILING_WHITESPACE, b"V", HeaderFlags::FOLDING)))))]
#[case::non_token(b"K\x0c:Value\r\n V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K\x0c", HeaderFlags::NAME_NON_TOKEN_CHARS, b"Value V", HeaderFlags::FOLDING))), None)]
#[case::non_token_trailing(b"K\r :Value\r\n V\r\n\r\n", Ok((b!("\r\n"), Header::new_with_flags(b"K\r ", HeaderFlags::NAME_TRAILING_WHITESPACE | HeaderFlags::NAME_NON_TOKEN_CHARS, b"Value V", HeaderFlags::FOLDING))), None)]
fn test_header(
#[case] input: &[u8], #[case] expected: IResult<&[u8], Header>,
#[case] diff_res_expected: Option<IResult<&[u8], Header>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.header()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.header()(input), res_expected);
} else {
assert_eq!(res_parser.header()(input), expected);
}
}
#[rstest]
#[case::not_a_separator(b"\n", Err(Error(NomError::new(b!("\n"), Tag))), None)]
#[case::colon(b":value", Ok((b!("value"), 0)), None)]
#[case::colon_whitespace(b": value", Ok((b!("value"), 0)), None)]
#[case::colon_tab(b":\t value", Ok((b!("value"), 0)), None)]
fn test_separators(
#[case] input: &[u8], #[case] expected: IResult<&[u8], u64>,
#[case] diff_res_expected: Option<IResult<&[u8], u64>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.separator()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.separator()(input), res_expected);
} else {
assert_eq!(res_parser.separator()(input), expected);
}
}
#[rstest]
#[case::incomplete(b"name", Err(Incomplete(Needed::new(1))))]
#[case::token(b"name:", Ok((b!(":"), (b!(""), b!("name"), b!("")))))]
#[case::trailing_whitespace(b"name :", Ok((b!(":"), (b!(""), b!("name"), b!(" ")))))]
#[case::surrounding_whitespace(b" name :", Ok((b!(":"), (b!(" "), b!("name"), b!(" ")))))]
fn test_token_chars(#[case] input: &[u8], #[case] expected: IResult<&[u8], SurroundedBytes>) {
assert_eq!(token_chars(input), expected);
}
#[rstest]
#[case::name(b"Hello: world", Ok((b!(": world"), Name {name: b"Hello".to_vec(), flags: 0})), None)]
#[case::name(b"Host:www.google.com\rName: Value", Ok((b!(":www.google.com\rName: Value"), Name {name: b"Host".to_vec(), flags: 0})), None)]
#[case::trailing_whitespace(b"Hello : world", Ok((b!(": world"), Name {name: b"Hello".to_vec(), flags: HeaderFlags::NAME_TRAILING_WHITESPACE})), None)]
#[case::surrounding_whitespace(b" Hello : world", Ok((b!(": world"), Name {name: b"Hello".to_vec(), flags: HeaderFlags::NAME_LEADING_WHITESPACE | HeaderFlags::NAME_TRAILING_WHITESPACE})), None)]
#[case::semicolon(b"Hello;invalid: world", Ok((b!(": world"), Name {name: b"Hello;invalid".to_vec(), flags: HeaderFlags::NAME_NON_TOKEN_CHARS})), None)]
#[case::space(b"Hello invalid: world", Ok((b!(": world"), Name {name: b"Hello invalid".to_vec(), flags: HeaderFlags::NAME_NON_TOKEN_CHARS})), None)]
#[case::surrounding_internal_space(b" Hello invalid : world", Ok((b!(": world"), Name {name: b"Hello invalid".to_vec(), flags: HeaderFlags::NAME_LEADING_WHITESPACE | HeaderFlags::NAME_TRAILING_WHITESPACE | HeaderFlags::NAME_NON_TOKEN_CHARS})), None)]
#[case::only_space_name(b" : world", Ok((b!(": world"), Name {name: b"".to_vec(), flags: HeaderFlags::NAME_LEADING_WHITESPACE | HeaderFlags::NAME_TRAILING_WHITESPACE })), None)]
fn test_name(
#[case] input: &[u8], #[case] expected: IResult<&[u8], Name>,
#[case] diff_res_expected: Option<IResult<&[u8], Name>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.name()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.name()(input), res_expected);
} else {
assert_eq!(res_parser.name()(input), expected);
}
}
#[rstest]
#[case(b"test", Err(Error(NomError::new(b!("test"), Tag))))]
#[case(b"\r\n", Err(Error(NomError::new(b!("\r\n"), Tag))))]
#[case(b"\n", Err(Error(NomError::new(b!("\n"), Tag))))]
#[case(b"\0a", Ok((b!("a"), (b!("\0"), HeaderFlags::NULL_TERMINATED))))]
fn test_null(#[case] input: &[u8], #[case] expected: IResult<&[u8], ParsedBytes>) {
assert_eq!(null(input), expected);
}
#[rstest]
#[case::not_eol(b"test", Err(Error(NomError::new(b!("test"), Tag))), None)]
#[case::incomplete_eol(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_eol(b"\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_eol(b"\r\n\t", Err(Error(NomError::new(b!("\t"), Not))), None)]
#[case::complete_cr(b"\ra", Err(Error(NomError::new(b!("\ra"), Tag))), Some(Ok((b!("a"), (b!("\r"), 0)))))]
#[case::incomplete_crcr(b"\r\r", Err(Error(NomError::new(b!("\r\r"), Tag))), Some(Ok((b!("\r"), (b!("\r"), 0)))))]
#[case::incomplete_lfcr(b"\n\r", Ok((b!("\r"), (b!("\n"), 0))), Some(Err(Incomplete(Needed::new(1)))))]
#[case::complete_lfcr(b"\n\ra", Ok((b!("\ra"), (b!("\n"), 0))), Some(Ok((b!("a"), (b!("\n\r"), 0)))))]
#[case::lfcrlf(b"\n\r\n", Ok((b!("\r\n"), (b!("\n"), 0))), Some(Ok((b!("\n"), (b!("\n\r"), 0)))))]
#[case::lfcrlfcr(b"\n\r\n\r", Ok((b!("\r\n\r"), (b!("\n"), 0))), Some(Ok((b!("\n\r"), (b!("\n\r"), 0)))))]
#[case::complete_lf(b"\na", Ok((b!("a"), (b!("\n"), 0))), None)]
#[case::complete_lfcrcrlf(b"\n\r\r\na", Ok((b!("\r\na"), (b!("\n\r"), HeaderFlags::DEFORMED_EOL))), Some(Ok((b!("\r\na"), (b!("\n\r"), 0)))))]
#[case::complete_crlfcrlf(b"\r\n\r\na", Ok((b!("\r\na"), (b!("\r\n"), 0))), None)]
#[case::incomplete_crlf(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_lf(b"\n", Err(Incomplete(Needed::new(1))), None)]
#[case::lfcrcrlf(b"\n\r\r\n", Ok((b!("\r\n"), (b!("\n\r"), HeaderFlags::DEFORMED_EOL))), Some(Ok((b!("\r\n"), (b!("\n\r"), 0)))))]
#[case::crlfcrlf(b"\r\n\r\n", Ok((b!("\r\n"), (b!("\r\n"), 0))), None)]
#[case::null(b"\0a", Err(Error(NomError::new(b!("\0a"), Tag))), None)]
fn test_eol(
#[case] input: &[u8], #[case] expected: IResult<&[u8], ParsedBytes>,
#[case] diff_res_expected: Option<IResult<&[u8], ParsedBytes>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.eol()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.eol()(input), res_expected);
} else {
assert_eq!(res_parser.eol()(input), expected);
}
}
#[rstest]
#[case::not_eol(b"test", Err(Error(NomError::new(b!("test"), Tag))), None)]
#[case::incomplete_eol(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_eol(b"\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_eol(b"\r\n\t", Err(Error(NomError::new(b!("\t"), Not))), None)]
#[case::complete_cr(b"\ra", Err(Error(NomError::new(b!("\ra"), Tag))), Some(Ok((b!("a"), (b!("\r"), 0)))))]
#[case::incomplete_crcr(b"\r\r", Err(Error(NomError::new(b!("\r\r"), Tag))), Some(Ok((b!("\r"), (b!("\r"), 0)))))]
#[case::incomplete_lfcr(b"\n\r", Ok((b!("\r"), (b!("\n"), 0))), Some(Err(Incomplete(Needed::new(1)))))]
#[case::complete_lfcr(b"\n\ra", Ok((b!("\ra"), (b!("\n"), 0))), Some(Ok((b!("a"), (b!("\n\r"), 0)))))]
#[case::lfcrlf(b"\n\r\n", Ok((b!("\r\n"), (b!("\n"), 0))), Some(Ok((b!("\n"), (b!("\n\r"), 0)))))]
#[case::lfcrlfcr(b"\n\r\n\r", Ok((b!("\r\n\r"), (b!("\n"), 0))), Some(Ok((b!("\n\r"), (b!("\n\r"), 0)))))]
#[case::complete_lf(b"\na", Ok((b!("a"), (b!("\n"), 0))), None)]
#[case::complete_lfcrcrlf(b"\n\r\r\na", Ok((b!("\r\na"), (b!("\n\r"), HeaderFlags::DEFORMED_EOL))), Some(Ok((b!("\r\na"), (b!("\n\r"), 0)))))]
#[case::complete_crlfcrlf(b"\r\n\r\na", Ok((b!("\r\na"), (b!("\r\n"), 0))), None)]
#[case::incomplete_crlf(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_lf(b"\n", Err(Incomplete(Needed::new(1))), None)]
#[case::lfcrcrlf(b"\n\r\r\n", Ok((b!("\r\n"), (b!("\n\r"), HeaderFlags::DEFORMED_EOL))), Some(Ok((b!("\r\n"), (b!("\n\r"), 0)))))]
#[case::crlfcrlf(b"\r\n\r\n", Ok((b!("\r\n"), (b!("\r\n"), 0))), None)]
#[case::null(b"\0a", Ok((b!("a"), (b!("\0"), HeaderFlags::NULL_TERMINATED))), None)]
fn test_null_or_eol(
#[case] input: &[u8], #[case] expected: IResult<&[u8], ParsedBytes>,
#[case] diff_res_expected: Option<IResult<&[u8], ParsedBytes>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.null_or_eol()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.null_or_eol()(input), res_expected);
} else {
assert_eq!(res_parser.null_or_eol()(input), expected);
}
}
#[rstest]
#[case::no_fold_tag(b"test", Err(Error(NomError::new(b!("test"), Tag))), None)]
#[case::cr(b"\r", Err(Error(NomError::new(b!("\r"), Tag))), Some(Err(Incomplete(Needed::new(1)))))]
#[case::crcr(b"\r\r", Err(Error(NomError::new(b!("\r\r"), Tag))), Some(Err(Error(NomError::new(b!("\r"), Tag)))))]
#[case::incomplete_crlf(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_crlf_ws(b"\r\n\t", Ok((b!(""), (b!("\r\n"), b!("\t"), HeaderFlags::FOLDING))), None)]
#[case::incomplete_crlf_ws(b"\r\n \t", Ok((b!("\t"), (b!("\r\n"), b!(" "), HeaderFlags::FOLDING))), None)]
#[case::incomplete_crlfcr(b"\r\n\r", Err(Error(NomError::new(b!("\r"), Tag))), None)]
#[case::not_fold_1(b"\r\n\r\n", Err(Error(NomError::new(b!("\r\n"), Tag))), None)]
#[case::not_fold_2(b"\r\n\r\r", Err(Error(NomError::new(b!("\r\r"), Tag))), None)]
#[case::fold(b"\r\n next", Ok((b!("next"), (b!("\r\n"), b!(" "), HeaderFlags::FOLDING))), None)]
#[case::fold(b"\r\n\tnext", Ok((b!("next"), (b!("\r\n"), b!("\t"), HeaderFlags::FOLDING))), None)]
#[case::fold(b"\r\n\t next", Ok((b!(" next"), (b!("\r\n"), b!("\t"), HeaderFlags::FOLDING))), None)]
#[case::fold_not_res(b"\r\n\t\t\r\n", Ok((b!("\t\r\n"), (b!("\r\n"), b!("\t"), HeaderFlags::FOLDING))), None)]
#[case::fold_not_res(b"\r\n\t \t\r", Ok((b!(" \t\r"), (b!("\r\n"), b!("\t"), HeaderFlags::FOLDING))), None)]
#[case::fold_not_res(b"\r\n \n", Ok((b!(" \n"), (b!("\r\n"), b!(" "), HeaderFlags::FOLDING))), None)]
#[case::special_fold_not_res(b"\n\r \n", Err(Error(NomError::new(b!("\r \n"), Tag))), Some( Ok((b!(" \n"), (b!("\n\r"), b!(" "), HeaderFlags::FOLDING)))))]
#[case::special_fold_1(b"\r\n\rnext", Err(Error(NomError::new(b!("\rnext"), Tag))), None)]
#[case::special_fold_2(b"\r\n\r\t next", Err(Error(NomError::new(b!("\r\t next"), Tag))), None)]
#[case::fold_res(b"\r hello \n", Err(Error(NomError::new(b!("\r hello \n"), Tag))), Some(Ok((b!(" hello \n"), (b!("\r"), b!(" "), HeaderFlags::FOLDING)))))]
fn test_folding(
#[case] input: &[u8], #[case] expected: IResult<&[u8], FoldingBytes>,
#[case] diff_res_expected: Option<IResult<&[u8], FoldingBytes>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.folding()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.folding()(input), res_expected);
} else {
assert_eq!(res_parser.folding()(input), expected);
}
}
#[rstest]
#[case::incomplete_1(b"\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_2(b"\r\n\t", Ok((b!(""), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!("\t"))))), None)]
#[case::incomplete_3(b"\r\n ", Ok((b!(""), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!(" "))))), None)]
#[case::incomplete_4(b"\r\n\r", Ok((b!("\r"),((b!("\r\n"), 0), None))), Some(Err(Incomplete(Needed::new(1)))))]
#[case::crcr(b"\r\r", Err(Error(NomError::new(b!("\r\r"), Tag))), Some(Ok((b!("\r"), ((b!("\r"), 0), None)))))]
#[case::fold(b"\r\n\ta", Ok((b!("a"), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!("\t"))))), None)]
#[case::special_fold(b"\r\n\ra", Ok((b!("\ra"),((b!("\r\n"), 0), None))), None)]
#[case::fold(b"\r\n a", Ok((b!("a"), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!(" "))))), None)]
#[case::crlf_eol(b"\r\na", Ok((b!("a"), ((b!("\r\n"), 0), None))), None)]
#[case::lflf_eol(b"\n\na", Ok((b!("\na"), ((b!("\n"), 0), None))), None)]
#[case::crlfcrlf_eol(b"\r\n\r\na", Ok((b!("\r\na"), ((b!("\r\n"), 0), None))), None)]
#[case::req_deformed_eol(b"\n\r\r\na", Ok((b!("\r\na"), ((b!("\n\r"), HeaderFlags::DEFORMED_EOL), None))), Some(Ok((b!("\r\na"), ((b!("\n\r"), 0), None)))))]
#[case::null_terminated(b"\0a", Ok((b!("a"), ((b!("\0"), HeaderFlags::NULL_TERMINATED), None))), None)]
#[case::res_fold(b"\r a", Err(Error(NomError::new(b!("\r a"), Tag))), Some(Ok((b!("a"), ((b!("\r"), HeaderFlags::FOLDING), Some(b!(" ")))))))]
#[case::multi_space_line(b"\n \r\n\n", Ok((b!(" \r\n\n"), ((b!("\n"), HeaderFlags::FOLDING), Some(b!(" "))))), None)]
fn test_folding_or_terminator(
#[case] input: &[u8], #[case] expected: IResult<&[u8], FoldingOrTerminator>,
#[case] diff_res_expected: Option<IResult<&[u8], FoldingOrTerminator>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.folding_or_terminator()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.folding_or_terminator()(input), res_expected);
} else {
assert_eq!(res_parser.folding_or_terminator()(input), expected);
}
}
#[rstest]
#[case::incomplete_1(b" ", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_2(b"value", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_3(b"\tvalue", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_4(b" value", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_5(b"value\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete_6(b"\r\r", Err(Incomplete(Needed::new(1))), Some(Ok((b!("\r"), (b!(""), ((b!("\r"), 0), None))))))]
#[case::diff_values_1(b"www.google.com\rName: Value\r\n\r\n", Ok((b!("\r\n"), (b!("www.google.com\rName: Value"), ((b!("\r\n"), 0), None)))), Some(Ok((b!("Name: Value\r\n\r\n"), (b!("www.google.com"), ((b!("\r"), 0), None))))))]
#[case::diff_values_2(b"www.google.com\rName: Value\n\r\n", Ok((b!("\r\n"), (b!("www.google.com\rName: Value"), ((b!("\n"), 0), None)))), Some(Ok((b!("Name: Value\n\r\n"), (b!("www.google.com"), ((b!("\r"), 0), None))))))]
#[case::diff_values_3(b"www.google.com\rName: Value\r\n\n", Ok((b!("\n"), (b!("www.google.com\rName: Value"), ((b!("\r\n"), 0), None)))), Some(Ok((b!("Name: Value\r\n\n"), (b!("www.google.com"), ((b!("\r"), 0), None))))))]
#[case::value_1(b"\r\nnext", Ok((b!("next"), (b!(""), ((b!("\r\n"), 0), None)))), None)]
#[case::value_2(b"value\r\nname2", Ok((b!("name2"), (b!("value"), ((b!("\r\n"), 0), None)))), None)]
#[case::fold_value_1(b"value\n more", Ok((b!("more"), (b!("value"), ((b!("\n"), HeaderFlags::FOLDING), Some(b!(" ")))))), None)]
#[case::fold_value_2(b"value\r\n\t more", Ok((b!(" more"), (b!("value"), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!("\t")))))), None)]
#[case::req_special_fold_res_value_1(b"value\r\n\t more", Ok((b!(" more"), (b!("value"), ((b!("\r\n"), HeaderFlags::FOLDING), Some(b!("\t")))))), None)]
#[case::req_special_fold_res_value_2(b"value\n\rmore", Ok((b!("\rmore"), (b!("value"), ((b!("\n"), 0), None)))), Some(Ok((b!("more"), (b!("value"), ((b!("\n\r"), 0), None))))))]
#[case::special_fold(b"value\r\n\rmore", Ok((b!("\rmore"), (b!("value"), ((b!("\r\n"), 0), None)))), None)]
fn test_value_bytes(
#[case] input: &[u8], #[case] expected: IResult<&[u8], ValueBytes>,
#[case] diff_res_expected: Option<IResult<&[u8], ValueBytes>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.value_bytes()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.value_bytes()(input), res_expected);
} else {
assert_eq!(res_parser.value_bytes()(input), expected);
}
}
#[rstest]
#[case::incomplete(b"value\r\n more\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete(b"value\r\n ", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete(b"value\r\n more", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete(b"value\r\n more\n", Err(Incomplete(Needed::new(1))), None)]
#[case::incomplete(b"value\n more\r\n", Err(Incomplete(Needed::new(1))), None)]
#[case::fold(b"\r\n value \r\nnext:", Ok((b!("next:"), Value {value: b"value".to_vec(), flags: HeaderFlags::FOLDING})), None)]
#[case::fold(b"\r\n value\r\nnext:", Ok((b!("next:"), Value {value: b"value".to_vec(), flags: HeaderFlags::FOLDING})), None)]
#[case::fold(b"value\r\n more\r\n\r\n", Ok((b!("\r\n"), Value {value: b"value more".to_vec(), flags: HeaderFlags::FOLDING})), None)]
#[case::fold(b"value\r\n more\r\n\tand more\r\nnext:", Ok((b!("next:"), Value {value: b"value more\tand more".to_vec(), flags: HeaderFlags::FOLDING})), None)]
#[case::fold(b"value\n\t\tmore\r\n and\r\n more\r\nnext:", Ok((b!("next:"), Value {value: b"value\t\tmore and more".to_vec(), flags: HeaderFlags::FOLDING})), None)]
#[case::req_special_res_fold_1(b"value\n more\n\r\tand more\r\n\r\n", Ok((b!("\r\tand more\r\n\r\n"), Value {value: b"value more".to_vec(), flags: HeaderFlags::FOLDING})), Some(Ok((b!("\r\n"), Value {value: b"value more\tand more".to_vec(), flags: HeaderFlags::FOLDING}))))]
#[case::req_special_res_fold_2(b"value\n\r\t\tmore\r\n and\r\n more\r\nnext:", Ok((b!("\r\t\tmore\r\n and\r\n more\r\nnext:"), Value {value: b"value".to_vec(), flags: 0})), Some(Ok((b!("next:"), Value {value: b"value\t\tmore and more".to_vec(), flags: HeaderFlags::FOLDING}))))]
#[case::req_special_res_value(b"value\n\r\t\tmore\r\n and\r\n more\r\nnext:", Ok((b!("\r\t\tmore\r\n and\r\n more\r\nnext:"), Value {value: b"value".to_vec(), flags: 0})), Some(Ok((b!("next:"), Value {value: b"value\t\tmore and more".to_vec(), flags: HeaderFlags::FOLDING}))))]
#[case::req_special_deformed_res_fold(b"value1\n\r next: value2\r\n and\r\n more\r\nnext3:", Ok((b!("\r next: value2\r\n and\r\n more\r\nnext3:"), Value {value: b"value1".to_vec(), flags: 0})), Some(Ok((b!("next: value2\r\n and\r\n more\r\nnext3:"), Value {value: b"value1".to_vec(), flags: 0}))))]
#[case::value(b"value\r\nnext:", Ok((b!("next:"), Value {value: b"value".to_vec(), flags: 0})), None)]
#[case::value_empty(b"\r\nnext:", Ok((b!("next:"), Value {value: b"".to_vec(), flags: HeaderFlags::VALUE_EMPTY})), None)]
#[case::value_wrapping_with_colon(b"b\r\n c: d\r\nAAA", Ok((b!("AAA"), Value {value: b"b c: d".to_vec(), flags: HeaderFlags::FOLDING})), Some(Ok((b!("c: d\r\nAAA"), Value {value: b"b".to_vec(), flags: 0}))))]
#[case::value_wrapping_with_colon_no_tokens(b"b\r\n : d\r\nAAA", Ok((b!("AAA"), Value {value: b"b : d".to_vec(), flags: HeaderFlags::FOLDING})), Some(Ok((b!("AAA"), Value {value: b"b : d".to_vec(), flags: HeaderFlags::FOLDING}))))]
fn test_value(
#[case] input: &[u8], #[case] expected: IResult<&[u8], Value>,
#[case] diff_res_expected: Option<IResult<&[u8], Value>>,
) {
let req_parser = Parser::new(Side::Request);
assert_eq!(req_parser.value()(input), expected);
let res_parser = Parser::new(Side::Response);
if let Some(res_expected) = diff_res_expected {
assert_eq!(res_parser.value()(input), res_expected);
} else {
assert_eq!(res_parser.value()(input), expected);
}
}
}
rust/htp/src/hook.rs
+120
View File
@@ -0,0 +1,120 @@
use crate::{
connection_parser::{ConnectionParser, ParserData},
error::Result,
transaction::{Data, Transaction},
HtpStatus,
};
/// External (C) callback function prototype
pub(crate) type TxExternalCallbackFn =
unsafe extern "C" fn(connp: *const ConnectionParser, tx: *mut Transaction) -> HtpStatus;
/// Native (rust) callback function prototype
pub(crate) type TxNativeCallbackFn = fn(tx: &mut Transaction) -> Result<()>;
/// Hook for Transaction
pub(crate) type TxHook = Hook<TxExternalCallbackFn, TxNativeCallbackFn>;
/// External (C) callback function prototype
pub(crate) type DataExternalCallbackFn =
unsafe extern "C" fn(connp: *const ConnectionParser, data: *mut Data) -> HtpStatus;
/// Native (rust) callback function prototype
pub(crate) type DataNativeCallbackFn = fn(&mut Transaction, data: &ParserData) -> Result<()>;
/// Hook for Data
pub(crate) type DataHook = Hook<DataExternalCallbackFn, DataNativeCallbackFn>;
/// Callback list
#[derive(Clone)]
pub struct Hook<E, N> {
/// List of all callbacks.
pub(crate) callbacks: Vec<Callback<E, N>>,
}
impl<E, N> Default for Hook<E, N> {
/// Create a new callback list
fn default() -> Self {
Hook {
callbacks: Vec::new(),
}
}
}
impl<E, N> Hook<E, N> {
/// Register a native (rust) callback function
#[cfg(test)]
pub(crate) fn register(&mut self, cbk_fn: N) {
self.callbacks.push(Callback::Native(cbk_fn))
}
/// Register an external (C) callback function
pub(crate) fn register_extern(&mut self, cbk_fn: E) {
self.callbacks.push(Callback::External(cbk_fn))
}
}
impl TxHook {
/// Run all callbacks on the list
///
/// This function will exit early if a callback fails to return HtpStatus::OK
/// or HtpStatus::DECLINED.
pub(crate) fn run_all(&self, connp: &mut ConnectionParser, tx_index: usize) -> Result<()> {
let connp_ptr: *mut ConnectionParser = connp as *mut ConnectionParser;
if let Some(tx) = connp.tx_mut(tx_index) {
for cbk_fn in &self.callbacks {
match cbk_fn {
Callback::External(cbk_fn) => {
let result = unsafe { cbk_fn(connp_ptr, tx) };
if result != HtpStatus::OK && result != HtpStatus::DECLINED {
return Err(result);
}
}
Callback::Native(cbk_fn) => {
if let Err(e) = cbk_fn(tx) {
if e != HtpStatus::DECLINED {
return Err(e);
}
}
}
};
}
}
Ok(())
}
}
impl DataHook {
/// Run all callbacks on the list
///
/// This function will exit early if a callback fails to return HtpStatus::OK
/// or HtpStatus::DECLINED.
pub(crate) fn run_all(&self, connp: &ConnectionParser, data: &mut Data) -> Result<()> {
for cbk_fn in &self.callbacks {
match cbk_fn {
Callback::External(cbk_fn) => {
let result = unsafe { cbk_fn(connp, data) };
if result != HtpStatus::OK && result != HtpStatus::DECLINED {
return Err(result);
}
}
Callback::Native(cbk_fn) => {
if let Err(e) = cbk_fn(unsafe { &mut *data.tx() }, data.parser_data()) {
if e != HtpStatus::DECLINED {
return Err(e);
}
}
}
};
}
Ok(())
}
}
/// Type of callbacks
#[derive(Copy, Clone)]
pub enum Callback<E, N> {
/// External (C) callback function
External(E),
/// Native (rust) callback function
Native(N),
}
rust/htp/src/lib.rs
+93
View File
@@ -0,0 +1,93 @@
//! Root crate for libhtp.
#![deny(warnings)]
#![deny(missing_docs)]
#![deny(unused_lifetimes)]
#![allow(non_camel_case_types)]
#![allow(non_snake_case)]
#![allow(non_upper_case_globals)]
#[repr(C)]
#[derive(PartialEq, Eq, Debug)]
/// Status codes used by LibHTP internally.
pub enum HtpStatus {
/// The lowest value LibHTP will use internally.
ERROR_RESERVED = -1000,
/// General-purpose error code.
ERROR = -1,
/// No processing or work was done. This is typically used by callbacks
/// to indicate that they were not interested in doing any work in the
/// given context.
DECLINED = 0,
/// Returned by a function when its work was successfully completed.
OK = 1,
/// Returned when processing a connection stream, after consuming all
/// provided data. The caller should call again with more data.
DATA = 2,
/// Returned when processing a connection stream, after encountering
/// a situation where processing needs to continue on the alternate
/// stream (e.g., the inbound parser needs to observe some outbound
/// data). The data provided was not completely consumed. On the next
/// invocation the caller should supply only the data that has not
/// been processed already. Use request_data_consumed() and response_data_consumed()
/// to determine how much of the most recent data chunk was consumed.
DATA_OTHER = 3,
/// Used by callbacks to indicate that the processing should stop. For example,
/// returning HtpStatus::STOP from a connection callback indicates that LibHTP should
/// stop following that particular connection.
STOP = 4,
/// Same as DATA, but indicates that any non-consumed part of the data chunk
/// should be preserved (buffered) for later.
DATA_BUFFER = 5,
/// The highest value LibHTP will use internally.
STATUS_RESERVED = 1000,
}
/// Module for providing logging functions.
#[macro_use]
pub mod log;
/// Module for bstr functions.
pub mod bstr;
/// Module for all functions facing c_api.
pub mod c_api;
/// Module for all decompressors functions.
pub mod decompressors;
/// Module for all errors.
pub mod error;
/// Module for header parsing.
mod headers;
/// Module for hooks.
pub mod hook;
/// Module for providing unicode bestfit mappings.
#[macro_use]
mod unicode_bestfit_map;
/// Module for libhtp configurations.
pub mod config;
/// Module for all connection.
pub mod connection;
/// Module for connection parser.
pub mod connection_parser;
/// Module for extra utility parsers. (only public for doc tests)
pub mod parsers;
/// Module for request parsing.
pub mod request;
/// Module for response parsing.
pub mod response;
/// Module for custom table.
pub mod table;
/// Module for transaction parsing.
pub mod transaction;
/// Module to track multiple transactions
pub mod transactions;
/// Module for uri parsing.
pub mod uri;
/// Module for url decoding.
pub mod urlencoded;
/// Module for utf8 decoding.
mod utf8_decoder;
/// Module for utility functions.
pub mod util;
/// Test harness
#[cfg(test)]
pub mod test;
rust/htp/src/log.rs
+293
View File
@@ -0,0 +1,293 @@
use std::cell::RefCell;
use std::collections::VecDeque;
use std::rc::Rc;
/// Different codes used for logging.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum HtpLogCode {
/// Default
UNKNOWN = 0,
/// Gzip Decompression Failed
GZIP_DECOMPRESSION_FAILED,
/// Request field missing a colon.
REQUEST_FIELD_MISSING_COLON,
/// Response field missing a colon.
RESPONSE_FIELD_MISSING_COLON,
/// Request chunk length parsing failed.
INVALID_REQUEST_CHUNK_LEN,
/// Response chunked-length parsing failed.
INVALID_RESPONSE_CHUNK_LEN,
/// Response chunk exension.
REQUEST_CHUNK_EXTENSION,
/// Response chunk exension.
RESPONSE_CHUNK_EXTENSION,
/// Request has too many headers.
REQUEST_TOO_MANY_HEADERS,
/// Response has too many headers.
RESPONSE_TOO_MANY_HEADERS,
/// Request transfer-encoding invalid.
INVALID_TRANSFER_ENCODING_VALUE_IN_REQUEST,
/// Response transfer-encoding invalid.
INVALID_TRANSFER_ENCODING_VALUE_IN_RESPONSE,
/// Request content-length parsing failed.
INVALID_CONTENT_LENGTH_FIELD_IN_REQUEST,
/// Response content-length parsing failed.
INVALID_CONTENT_LENGTH_FIELD_IN_RESPONSE,
/// Request has a duplicate content-length field.
DUPLICATE_CONTENT_LENGTH_FIELD_IN_REQUEST,
/// Response has a duplicate content-length field.
DUPLICATE_CONTENT_LENGTH_FIELD_IN_RESPONSE,
/// 100 Continue response status already seen.
CONTINUE_ALREADY_SEEN,
/// Unable to match response to a request.
UNABLE_TO_MATCH_RESPONSE_TO_REQUEST,
/// Request server port is invalid.
INVALID_SERVER_PORT_IN_REQUEST,
/// Authority port is invalid.
INVALID_AUTHORITY_PORT,
/// Request header name is incorrectly formed.
REQUEST_HEADER_INVALID,
/// Response header name is incorrectly formed.
RESPONSE_HEADER_INVALID,
/// Host header is missing.
MISSING_HOST_HEADER,
/// Host header is ambiguous.
HOST_HEADER_AMBIGUOUS,
/// Request has invalid line folding.
INVALID_REQUEST_FIELD_FOLDING,
/// Response has invalid line folding.
INVALID_RESPONSE_FIELD_FOLDING,
/// Request buffer field is over the limit.
REQUEST_FIELD_TOO_LONG,
/// Response buffer field is over the limit.
RESPONSE_FIELD_TOO_LONG,
/// Mismatch between request server port and tcp port.
REQUEST_SERVER_PORT_TCP_PORT_MISMATCH,
/// Uri hostname is invalid.
URI_HOST_INVALID,
/// Header hostname is invalid.
HEADER_HOST_INVALID,
/// Non compliant delimiter between method and URI in request line.
METHOD_DELIM_NON_COMPLIANT,
/// Parsed request-uri contains a non compliant delimiter.
URI_DELIM_NON_COMPLIANT,
/// Request line has leading whitespace.
REQUEST_LINE_LEADING_WHITESPACE,
/// Response content encoding lzma layers is greater than limit.
RESPONSE_TOO_MANY_LZMA_LAYERS,
/// Request content encoding lzma layers is greater than limit.
REQUEST_TOO_MANY_LZMA_LAYERS,
/// Too many request or response encoding layers
TOO_MANY_ENCODING_LAYERS,
/// Response header content-encoding header is invalid
ABNORMAL_CE_HEADER,
/// Request authorization header unrecognized
AUTH_UNRECOGNIZED,
/// Request header has been seen more than once.
REQUEST_HEADER_REPETITION,
/// response header has been seen more than once.
RESPONSE_HEADER_REPETITION,
/// Response content-type is multipart-byteranges (unsupported).
RESPONSE_MULTIPART_BYTERANGES,
/// Response transfer-encoding has an abnormal chunked value.
RESPONSE_ABNORMAL_TRANSFER_ENCODING,
/// Response chunked transfer-encoding on HTTP/0.9 or HTTP/1.0.
RESPONSE_CHUNKED_OLD_PROTO,
/// Response protocol invalid.
RESPONSE_INVALID_PROTOCOL,
/// Response status invalid.
RESPONSE_INVALID_STATUS,
/// Response line is incomplete.
REQUEST_LINE_INCOMPLETE,
/// Request uri has double encoding.
DOUBLE_ENCODED_URI,
/// Request line is invalid.
REQUEST_LINE_INVALID,
/// Unexpected request body present.
REQUEST_BODY_UNEXPECTED,
/// Reached LZMA memory limit.
LZMA_MEMLIMIT_REACHED,
/// Reached configured time limit for decompression or reached bomb limit.
COMPRESSION_BOMB,
/// Unexpected response body present.
RESPONSE_BODY_UNEXPECTED,
/// Content-length parsing contains extra leading characters.
CONTENT_LENGTH_EXTRA_DATA_START,
/// Content-length parsing contains extra trailing characters
CONTENT_LENGTH_EXTRA_DATA_END,
/// 101 Switching Protocol seen with a content-length.
SWITCHING_PROTO_WITH_CONTENT_LENGTH,
/// End of line is deformed.
DEFORMED_EOL,
/// Parsing error encountered in request or response.
PARSER_STATE_ERROR,
/// Missing outbound transaction while state is not idle.
MISSING_OUTBOUND_TRANSACTION_DATA,
/// Missing inbound transaction while state is not idle.
MISSING_INBOUND_TRANSACTION_DATA,
/// Supplied data chunk has a length of zero.
ZERO_LENGTH_DATA_CHUNKS,
/// Request Line method is unknown.
REQUEST_LINE_UNKNOWN_METHOD,
/// Request line method is unknown and no protocol information was found.
REQUEST_LINE_UNKNOWN_METHOD_NO_PROTOCOL,
/// Request line method is unknown and protocol is invalid.
REQUEST_LINE_UNKNOWN_METHOD_INVALID_PROTOCOL,
/// Request line protocol information was not found.
REQUEST_LINE_NO_PROTOCOL,
/// Response line protocol is invalid.
RESPONSE_LINE_INVALID_PROTOCOL,
/// Response line status number is out of range.
RESPONSE_LINE_INVALID_RESPONSE_STATUS,
/// Response parsing progress is at an invalid state.
RESPONSE_BODY_INTERNAL_ERROR,
/// Request body data callback produced a error.
REQUEST_BODY_DATA_CALLBACK_ERROR,
/// Response header name is empty.
RESPONSE_INVALID_EMPTY_NAME,
/// Request header name is empty.
REQUEST_INVALID_EMPTY_NAME,
/// Response header name has extra whitespace after name.
RESPONSE_INVALID_LWS_AFTER_NAME,
/// Response header name is not a valid token.
RESPONSE_HEADER_NAME_NOT_TOKEN,
/// Request header name has extra whitespace after name.
REQUEST_INVALID_LWS_AFTER_NAME,
/// LZMA decompression is disabled.
LZMA_DECOMPRESSION_DISABLED,
/// Tried to open a connection that is already open.
CONNECTION_ALREADY_OPEN,
/// Protocol parsing detected leading or trailing data.
PROTOCOL_CONTAINS_EXTRA_DATA,
/// Invalid gap detected.
INVALID_GAP,
/// Compression bomb due to double lzma encoding.
COMPRESSION_BOMB_DOUBLE_LZMA,
/// Invalid content-encoding detected.
INVALID_CONTENT_ENCODING,
/// Error retrieving a log message's code
ERROR,
}
/// Enumerates all log levels.
#[repr(C)]
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Debug)]
pub(crate) enum HtpLogLevel {
/// Designates fatal error.
Error,
/// Designates hazardous situations.
Warning,
/// Default log level value.
Notice,
/// Designates useful information,
Info,
}
#[derive(Clone)]
/// Logger struct
pub(crate) struct Logger {
/// The sender half of a logging channel
pub(crate) sender: Rc<RefCell<VecDeque<Log>>>,
/// Log level used when deciding whether to store or
/// ignore the messages issued by the parser.
pub(crate) level: HtpLogLevel,
}
impl Logger {
/// Returns a new logger instance
pub(crate) fn new(sender: &Rc<RefCell<VecDeque<Log>>>) -> Logger {
Self {
sender: sender.clone(),
level: HtpLogLevel::Notice,
}
}
/// Logs a message to the logger channel.
pub(crate) fn log(
&mut self, _file: &str, _line: u32, level: HtpLogLevel, code: HtpLogCode, msg: String,
) {
// Ignore messages below our log level.
if level <= self.level {
let mut sender = self.sender.borrow_mut();
sender.push_back(Log::new(Message::new(code, msg)));
}
}
}
#[derive(Clone)]
/// Represents a single Message entry for a log
pub(crate) struct Message {
/// Log message string.
pub(crate) msg: String,
//level: HtpLogLevel,
/// Message code.
pub(crate) code: HtpLogCode,
//pub(crate) file: String,
//line: u32,
}
impl Message {
/// Returns a new Message instance
pub(crate) fn new(code: HtpLogCode, msg: String) -> Message {
Self { code, msg }
}
}
/// Represents a single log entry.
#[derive(Clone)]
pub struct Log {
/// Log message.
pub(crate) msg: Message,
}
impl Log {
/// Returns a new Log instance.
pub(crate) fn new(msg: Message) -> Log {
Self { msg }
}
}
/// Logs a message at the given level.
#[macro_export]
macro_rules! htp_log {
($logger:expr, $level:expr, $code:expr, $msg:expr) => {{
use $crate::log::{HtpLogCode, HtpLogLevel};
$logger.log(file!(), line!(), $level, $code, $msg.to_string());
}};
}
/// Logs a message at the info level.
#[macro_export]
macro_rules! htp_info {
($logger:expr, $code:expr, $msg:expr) => {
htp_log!($logger, HtpLogLevel::Info, $code, $msg);
};
}
/// Logs a message at the warning level.
#[macro_export]
macro_rules! htp_warn {
($logger:expr, $code:expr, $msg:expr) => {
htp_log!($logger, HtpLogLevel::Warning, $code, $msg);
};
}
/// Logs a message at the error level.
#[macro_export]
macro_rules! htp_error {
($logger:expr, $code:expr, $msg:expr) => {
htp_log!($logger, HtpLogLevel::Error, $code, $msg);
};
}
/// Logs a message at the warning level, ensuring that it ones logs the message once.
#[macro_export]
macro_rules! htp_warn_once {
($logger:expr, $code:expr, $msg:expr, $tx_flags:expr, $flags:expr, $flag:expr) => {
// Log only once per transaction.
if !$tx_flags.is_set($flag) {
htp_warn!($logger, $code, $msg);
}
$tx_flags.set($flag);
$flags.set($flag);
};
}
rust/htp/src/parsers.rs
+629
View File
@@ -0,0 +1,629 @@
use crate::{
bstr::Bstr,
error::Result,
log::Logger,
transaction::{Header, HtpAuthType, HtpProtocol, HtpResponseNumber, Transaction},
util::{
ascii_digits, convert_port, hex_digits, take_ascii_whitespace, take_chunked_ctl_chars,
validate_hostname,
},
HtpStatus,
};
use base64::{engine::general_purpose::STANDARD, Engine};
use nom::{
branch::alt,
bytes::complete::{is_not, tag, tag_no_case, take_till, take_until, take_while},
combinator::{map, not, opt, peek},
error::ErrorKind,
multi::many0,
sequence::tuple,
IResult,
};
/// Parses the content type header, trimming any leading whitespace.
/// Finds the end of the MIME type, using the same approach PHP 5.4.3 uses.
///
/// Returns a tuple of the remaining unparsed header data and the content type
fn content_type() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
map(
tuple((
take_ascii_whitespace(),
take_till(|c| c == b';' || c == b',' || c == b' '),
)),
|(_, content_type)| content_type,
)(input)
}
}
/// Parses the content type header from the given header value, lowercases it, and stores it in the provided ct bstr.
/// Finds the end of the MIME type, using the same approach PHP 5.4.3 uses.
pub(crate) fn parse_content_type(header: &[u8]) -> Result<Bstr> {
let (_, content_type) = content_type()(header)?;
let mut ct = Bstr::from(content_type);
ct.make_ascii_lowercase();
Ok(ct)
}
/// Parses Content-Length string (positive decimal number). White space is
/// allowed before and after the number.
///
/// Returns content length, or None if input is not valid.
pub(crate) fn parse_content_length(input: &[u8], logger: Option<&mut Logger>) -> Option<u64> {
let (trailing_data, (leading_data, content_length)) = ascii_digits(input).ok()?;
if let Some(logger) = logger {
if !leading_data.is_empty() {
// Contains invalid characters! But still attempt to process
htp_warn!(
logger,
HtpLogCode::CONTENT_LENGTH_EXTRA_DATA_START,
"C-L value with extra data in the beginning"
);
}
if !trailing_data.is_empty() {
// Ok to have junk afterwards
htp_warn!(
logger,
HtpLogCode::CONTENT_LENGTH_EXTRA_DATA_END,
"C-L value with extra data in the end"
);
}
}
std::str::from_utf8(content_length)
.ok()?
.parse::<u64>()
.ok()
}
/// Parses chunked length (positive hexadecimal number). White space is allowed before
/// and after the number.
pub(crate) fn parse_chunked_length(input: &[u8]) -> Result<(Option<u64>, bool)> {
let (rest, _) = take_chunked_ctl_chars(input)?;
let (trailing_data, chunked_length) = hex_digits()(rest)?;
if trailing_data.is_empty() && chunked_length.is_empty() {
return Ok((None, false));
}
let chunked_len = u64::from_str_radix(
std::str::from_utf8(chunked_length).map_err(|_| HtpStatus::ERROR)?,
16,
)
.map_err(|_| HtpStatus::ERROR)?;
//TODO: remove this limit and update appropriate tests after differential fuzzing
if chunked_len > i32::MAX as u64 {
return Ok((None, false));
}
let has_ext = trailing_data.contains(&b';');
Ok((Some(chunked_len), has_ext))
}
/// Attempts to extract the scheme from a given input URI.
///
/// Returns a tuple of the unconsumed data and the matched scheme.
pub(crate) fn scheme() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
// Scheme test: if it doesn't start with a forward slash character (which it must
// for the contents to be a path or an authority), then it must be the scheme part
map(
tuple((peek(not(tag("/"))), take_until(":"), tag(":"))),
|(_, scheme, _)| scheme,
)(input)
}
}
/// Helper for parsed credentials (username, Option<password>)
pub(crate) type ParsedCredentials<'a> = (&'a [u8], Option<&'a [u8]>);
/// Attempts to extract the credentials from a given input URI, assuming the scheme has already been extracted.
///
/// Returns a tuple of the remaining unconsumed data and a tuple of the matched username and password.
pub(crate) fn credentials() -> impl Fn(&[u8]) -> IResult<&[u8], ParsedCredentials> {
move |input| {
// Authority test: two forward slash characters and it's an authority.
// One, three or more slash characters, and it's a path.
// Note: we only attempt to parse authority if we've seen a scheme.
let (input, (_, _, credentials, _)) =
tuple((tag("//"), peek(not(tag("/"))), take_until("@"), tag("@")))(input)?;
let (password, username) = opt(tuple((take_until(":"), tag(":"))))(credentials)?;
if let Some((username, _)) = username {
Ok((input, (username, Some(password))))
} else {
Ok((input, (credentials, None)))
}
}
}
/// Attempts to extract an IPv6 hostname from a given input URI,
/// assuming any scheme, credentials, hostname, port, and path have been already parsed out.
///
/// Returns a tuple of the remaining unconsumed data and the matched ipv6 hostname.
pub(crate) fn ipv6() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| -> IResult<&[u8], &[u8]> {
let (rest, _) = tuple((tag("["), is_not("/?#]"), opt(tag("]"))))(input)?;
Ok((rest, &input[..input.len() - rest.len()]))
}
}
/// Attempts to extract the hostname from a given input URI
///
/// Returns a tuple of the remaining unconsumed data and the matched hostname.
pub(crate) fn hostname() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
let (input, mut hostname) = map(
tuple((
opt(tag("//")), //If it starts with "//", skip (might have parsed a scheme and no creds)
peek(not(tag("/"))), //If it starts with '/', this is a path, not a hostname
many0(tag(" ")),
alt((ipv6(), is_not("/?#:"))),
)),
|(_, _, _, hostname)| hostname,
)(input)?;
//There may be spaces in the middle of a hostname, so much trim only at the end
while hostname.ends_with(b" ") {
hostname = &hostname[..hostname.len() - 1];
}
Ok((input, hostname))
}
}
/// Attempts to extract the port from a given input URI,
/// assuming any scheme, credentials, or hostname have been already parsed out.
///
/// Returns a tuple of the remaining unconsumed data and the matched port.
pub(crate) fn port() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
// Must start with ":" for there to be a port to parse
let (input, (_, _, port, _)) =
tuple((tag(":"), many0(tag(" ")), is_not("/?#"), many0(tag(" "))))(input)?;
let (_, port) = is_not(" ")(port)?; //we assume there never will be a space in the middle of a port
Ok((input, port))
}
}
/// Attempts to extract the path from a given input URI,
/// assuming any scheme, credentials, hostname, and port have been already parsed out.
///
/// Returns a tuple of the remaining unconsumed data and the matched path.
pub(crate) fn path() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| is_not("#?")(input)
}
/// Attempts to extract the query from a given input URI,
/// assuming any scheme, credentials, hostname, port, and path have been already parsed out.
///
/// Returns a tuple of the remaining unconsumed data and the matched query.
pub(crate) fn query() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
// Skip the starting '?'
map(tuple((tag("?"), take_till(|c| c == b'#'))), |(_, query)| {
query
})(input)
}
}
/// Attempts to extract the fragment from a given input URI,
/// assuming any other components have been parsed out.
///
/// Returns a tuple of the remaining unconsumed data and the matched fragment.
pub(crate) fn fragment() -> impl Fn(&[u8]) -> IResult<&[u8], &[u8]> {
move |input| {
// Skip the starting '#'
let (input, _) = tag("#")(input)?;
Ok((b"", input))
}
}
type parsed_port<'a> = Option<(&'a [u8], Option<u16>)>;
type parsed_hostport<'a> = (&'a [u8], parsed_port<'a>, bool);
/// Parses an authority string, which consists of a hostname with an optional port number
///
/// Returns a remaining unparsed data, parsed hostname, parsed port, converted port number,
/// and a flag indicating whether the parsed data is valid.
pub(crate) fn parse_hostport(input: &[u8]) -> IResult<&[u8], parsed_hostport> {
let (input, host) = hostname()(input)?;
let mut valid = validate_hostname(host);
if let Ok((_, p)) = port()(input) {
if let Some(port) = convert_port(p) {
return Ok((input, (host, Some((p, Some(port))), valid)));
} else {
return Ok((input, (host, Some((p, None)), false)));
}
} else if !input.is_empty() {
//Trailing data after the hostname that is invalid e.g. [::1]xxxxx
valid = false;
}
Ok((input, (host, None, valid)))
}
/// Extracts the version protocol from the input slice.
///
/// Returns (any unparsed trailing data, (version_number, flag indicating whether input contains trailing and/or leading whitespace and/or leading zeros))
fn protocol_version(input: &[u8]) -> IResult<&[u8], (&[u8], bool)> {
map(
tuple((
take_ascii_whitespace(),
tag_no_case("HTTP"),
take_ascii_whitespace(),
tag("/"),
take_while(|c: u8| c.is_ascii_whitespace() || c == b'0'),
alt((tag(".9"), tag("1.0"), tag("1.1"))),
take_ascii_whitespace(),
)),
|(_, _, leading, _, trailing, version, _)| {
(version, !leading.is_empty() || !trailing.is_empty())
},
)(input)
}
/// Determines protocol number from a textual representation (i.e., "HTTP/1.1"). This
/// function tries to be flexible, allowing whitespace before and after the forward slash,
/// as well as allowing leading zeros in the version number. If such leading/trailing
/// characters are discovered, however, a warning will be logged.
///
/// Returns HtpProtocol version or invalid.
pub(crate) fn parse_protocol(input: &[u8], logger: &mut Logger) -> HtpProtocol {
if let Ok((remaining, (version, contains_trailing))) = protocol_version(input) {
if !remaining.is_empty() {
return HtpProtocol::Invalid;
}
if contains_trailing {
htp_warn!(
logger,
HtpLogCode::PROTOCOL_CONTAINS_EXTRA_DATA,
"HtpProtocol version contains leading and/or trailing whitespace and/or leading zeros"
);
}
match version {
b".9" => HtpProtocol::V0_9,
b"1.0" => HtpProtocol::V1_0,
b"1.1" => HtpProtocol::V1_1,
_ => HtpProtocol::Invalid,
}
} else {
HtpProtocol::Invalid
}
}
/// Determines the numerical value of a response status given as a string.
pub(crate) fn parse_status(status: &[u8]) -> HtpResponseNumber {
if let Ok((trailing_data, (leading_data, status_code))) = ascii_digits(status) {
if !trailing_data.is_empty() || !leading_data.is_empty() {
//There are invalid characters in the status code
return HtpResponseNumber::Invalid;
}
if let Ok(status_code) = std::str::from_utf8(status_code) {
if let Ok(status_code) = status_code.parse::<u16>() {
if (100..=999).contains(&status_code) {
return HtpResponseNumber::Valid(status_code);
}
}
}
}
HtpResponseNumber::Invalid
}
/// Parses Digest Authorization request header.
fn parse_authorization_digest(auth_header_value: &[u8]) -> IResult<&[u8], Vec<u8>> {
// Extract the username
let (mut remaining_input, _) = tuple((
take_until("username="),
tag("username="),
take_ascii_whitespace(), // allow lws
tag("\""), // First character after LWS must be a double quote
))(auth_header_value)?;
let mut result = Vec::new();
// Unescape any escaped double quotes and find the closing quote
loop {
let (remaining, (auth_header, _)) = tuple((take_until("\""), tag("\"")))(remaining_input)?;
remaining_input = remaining;
result.extend_from_slice(auth_header);
if result.last() == Some(&(b'\\')) {
// Remove the escape and push back the double quote
result.pop();
result.push(b'\"');
} else {
// We found the closing double quote!
break;
}
}
Ok((remaining_input, result))
}
/// Parses Basic Authorization request header.
fn parse_authorization_basic(request_tx: &mut Transaction, auth_header: &Header) -> Result<()> {
// Skip 'Basic<lws>'
let (remaining_input, _) =
tuple((tag_no_case("basic"), take_ascii_whitespace()))(auth_header.value.as_slice())
.map_err(|_| HtpStatus::DECLINED)?;
// Decode base64-encoded data
let decoded = STANDARD
.decode(remaining_input)
.map_err(|_| HtpStatus::DECLINED)?;
let (password, (username, _)) =
tuple::<_, _, (&[u8], ErrorKind), _>((take_until(":"), tag(":")))(decoded.as_slice())
.map_err(|_| HtpStatus::DECLINED)?;
request_tx.request_auth_username = Some(Bstr::from(username));
request_tx.request_auth_password = Some(Bstr::from(password));
Ok(())
}
/// Parses Authorization request header.
pub(crate) fn parse_authorization(request_tx: &mut Transaction) -> Result<()> {
let auth_header = if let Some(auth_header) = request_tx
.request_headers
.get_nocase_nozero("authorization")
{
auth_header.clone()
} else {
request_tx.request_auth_type = HtpAuthType::NONE;
return Ok(());
};
// TODO Need a flag to raise when failing to parse authentication headers.
if auth_header.value.starts_with_nocase("basic") {
// Basic authentication
request_tx.request_auth_type = HtpAuthType::BASIC;
return parse_authorization_basic(request_tx, &auth_header);
} else if auth_header.value.starts_with_nocase("digest") {
// Digest authentication
request_tx.request_auth_type = HtpAuthType::DIGEST;
let (_, auth_username) = parse_authorization_digest(auth_header.value.as_slice())
.map_err(|_| HtpStatus::DECLINED)?;
if let Some(username) = &mut request_tx.request_auth_username {
username.clear();
username.add(auth_username);
} else {
request_tx.request_auth_username = Some(Bstr::from(auth_username));
}
} else if auth_header.value.starts_with_nocase("bearer") {
request_tx.request_auth_type = HtpAuthType::BEARER;
let (token, _) = tuple((
tag_no_case("bearer"),
take_ascii_whitespace(), // allow lws
))(auth_header.value.as_slice())
.map_err(|_| HtpStatus::DECLINED)?;
request_tx.request_auth_token = Some(Bstr::from(token));
} else {
// Unrecognized authentication method
request_tx.request_auth_type = HtpAuthType::UNRECOGNIZED
}
Ok(())
}
#[cfg(test)]
mod test {
use super::*;
use rstest::rstest;
#[rstest]
#[case(" username= \"ivan\\\"r\\\"\"", "ivan\"r\"", "")]
#[case("username=\"ivan\\\"r\\\"\"", "ivan\"r\"", "")]
#[case("username=\"ivan\\\"r\\\"\" ", "ivan\"r\"", " ")]
#[case("username=\"ivanr\" ", "ivanr", " ")]
#[case("username= \"ivanr\" ", "ivanr", " ")]
#[should_panic]
#[case("username=ivanr\" ", "", "")]
#[should_panic]
#[case("username=\"ivanr ", "", "")]
fn test_parse_authorization_digest(
#[case] input: &str, #[case] username: &str, #[case] remaining: &str,
) {
assert_eq!(
parse_authorization_digest(input.as_bytes()).unwrap(),
(remaining.as_bytes(), username.as_bytes().to_vec())
);
}
#[rstest]
#[case(" 200 ", HtpResponseNumber::Valid(200))]
#[case(" \t 404 ", HtpResponseNumber::Valid(404))]
#[case("123", HtpResponseNumber::Valid(123))]
#[case("99", HtpResponseNumber::Invalid)]
#[case("1000", HtpResponseNumber::Invalid)]
#[case("200 OK", HtpResponseNumber::Invalid)]
#[case("NOT 200", HtpResponseNumber::Invalid)]
fn test_parse_status(#[case] input: &str, #[case] expected: HtpResponseNumber) {
assert_eq!(parse_status(&Bstr::from(input)), expected);
}
#[rstest]
#[case(
"http://user:[email protected]:1234/path1/path2?a=b&c=d#frag",
"http",
"//user:[email protected]:1234/path1/path2?a=b&c=d#frag"
)]
#[should_panic]
#[case(
"/http://user:[email protected]:1234/path1/path2?a=b&c=d#frag",
"",
""
)]
fn test_scheme(#[case] input: &str, #[case] s: &str, #[case] remaining: &str) {
assert_eq!(
scheme()(input.as_bytes()).unwrap(),
(remaining.as_bytes(), s.as_bytes())
);
}
#[rstest]
#[case(
"//user:[email protected]:1234/path1/path2?a=b&c=d#frag",
"user",
Some("pass"),
"www.example.com:1234/path1/path2?a=b&c=d#frag"
)]
#[case(
"//[email protected]:1234/path1/path2?a=b&c=d#frag",
"user",
None,
"www.example.com:1234/path1/path2?a=b&c=d#frag"
)]
#[should_panic]
#[case(
"http://user:[email protected]:1234/path1/path2?a=b&c=d#frag",
"",
None,
""
)]
fn test_credentials(
#[case] input: &str, #[case] username: &str, #[case] password: Option<&str>,
#[case] remaining: &str,
) {
assert_eq!(
credentials()(input.as_bytes()).unwrap(),
(
remaining.as_bytes(),
(username.as_bytes(), password.map(|i| i.as_bytes()))
)
);
}
#[rstest]
#[case(
"www.example.com:1234/path1/path2?a=b&c=d#frag",
"www.example.com",
":1234/path1/path2?a=b&c=d#frag"
)]
#[case(
"www.example.com/path1/path2?a=b&c=d#frag",
"www.example.com",
"/path1/path2?a=b&c=d#frag"
)]
#[case("www.example.com?a=b&c=d#frag", "www.example.com", "?a=b&c=d#frag")]
#[case("www.example.com#frag", "www.example.com", "#frag")]
#[case("[::1]:8080", "[::1]", ":8080")]
#[case("[::1", "[::1", "")]
#[case("[::1/path1[0]", "[::1", "/path1[0]")]
#[case("[::1]xxxx", "[::1]", "xxxx")]
#[should_panic]
#[case("/www.example.com/path1/path2?a=b&c=d#frag", "", "")]
fn test_hostname(#[case] input: &str, #[case] host: &str, #[case] remaining: &str) {
assert_eq!(
hostname()(input.as_bytes()).unwrap(),
(remaining.as_bytes(), host.as_bytes())
);
}
#[rstest]
#[case(":1234/path1/path2?a=b&c=d#frag", "1234", "/path1/path2?a=b&c=d#frag")]
#[case(":1234?a=b&c=d#frag", "1234", "?a=b&c=d#frag")]
#[case(":1234#frag", "1234", "#frag")]
#[should_panic]
#[case("1234/path1/path2?a=b&c=d#frag", "", "")]
fn test_port(#[case] input: &str, #[case] p: &str, #[case] remaining: &str) {
assert_eq!(
port()(input.as_bytes()).unwrap(),
(remaining.as_bytes(), p.as_bytes())
);
}
#[rstest]
#[case("/path1/path2?a=b&c=d#frag", "/path1/path2", "?a=b&c=d#frag")]
#[case("/path1/path2#frag", "/path1/path2", "#frag")]
#[case("path1/path2?a=b&c=d#frag", "path1/path2", "?a=b&c=d#frag")]
#[case("//", "//", "")]
#[case(
"/uid=0(root) gid=0(root) groups=0(root)asdf",
"/uid=0(root) gid=0(root) groups=0(root)asdf",
""
)]
fn test_path(#[case] input: &str, #[case] p: &str, #[case] remaining: &str) {
assert_eq!(
path()(input.as_bytes()).unwrap(),
(remaining.as_bytes(), p.as_bytes())
);
}
#[rstest]
#[case("?a=b&c=d#frag", "a=b&c=d", "#frag")]
#[case("?a=b&c=d", "a=b&c=d", "")]
#[case("?", "", "")]
fn test_query(#[case] input: &str, #[case] q: &str, #[case] remaining: &str) {
assert_eq!(
query()(input.as_bytes()).unwrap(),
(remaining.as_bytes(), q.as_bytes())
);
}
#[rstest]
#[case("#frag", "frag")]
#[case("##frag", "#frag")]
#[should_panic]
#[case("frag", "")]
#[should_panic]
#[case("/path#frag", "")]
fn test_fragment(#[case] input: &str, #[case] frag: &str) {
assert_eq!(
fragment()(input.as_bytes()).unwrap(),
("".as_bytes(), frag.as_bytes())
);
}
#[rstest]
#[case("www.example.com", "www.example.com", None, true, "")]
#[case(" www.example.com ", "www.example.com", None, true, "")]
#[case(" www.example.com:8001 ", "www.example.com", Some(("8001", Some(8001))), true, ":8001 ")]
#[case(" www.example.com : 8001 ", "www.example.com", Some(("8001", Some(8001))), true, ": 8001 ")]
#[case("www.example.com.", "www.example.com.", None, true, "")]
#[case("www.example.com.", "www.example.com.", None, true, "")]
#[case("www.example.com:", "www.example.com", None, false, ":")]
#[case("www.example.com:ff", "www.example.com", Some(("ff", None)), false, ":ff")]
#[case("www.example.com:0", "www.example.com", Some(("0", None)), false, ":0")]
#[case("www.example.com:65536", "www.example.com", Some(("65536", None)), false, ":65536")]
#[case("[::1]:8080", "[::1]", Some(("8080", Some(8080))), true, ":8080")]
#[case("[::1]:", "[::1]", None, false, ":")]
#[case("[::1]x", "[::1]", None, false, "x")]
#[case("[::1", "[::1", None, false, "")]
fn test_parse_hostport(
#[case] input: &str, #[case] hostname: &str,
#[case] parsed_port: Option<(&str, Option<u16>)>, #[case] valid: bool,
#[case] remaining: &str,
) {
assert_eq!(
parse_hostport(input.as_bytes()).unwrap(),
(
remaining.as_bytes(),
(
hostname.as_bytes(),
parsed_port.map(|(port, port_nmb)| (port.as_bytes(), port_nmb)),
valid
)
)
);
}
#[rstest]
#[case("134", Some(134))]
#[case(" \t134 ", Some(134))]
#[case("abcd134 ", Some(134))]
#[case("abcd ", None)]
fn test_parse_content_length(#[case] input: &str, #[case] expected: Option<u64>) {
assert_eq!(parse_content_length(input.as_bytes(), None), expected);
}
#[rstest]
#[case("0 ; qw3=asd3; zc3=\"rt\"y3\"", (Some(0), true))]
#[case("12a5", (Some(0x12a5), false))]
#[case("12a5;ext=value", (Some(0x12a5), true))]
#[case(" \t12a5 ", (Some(0x12a5), false))]
#[case(" \t ", (None, false))]
fn test_parse_chunked_length(#[case] input: &str, #[case] expected: (Option<u64>, bool)) {
assert_eq!(parse_chunked_length(input.as_bytes()).unwrap(), expected);
}
#[rstest]
#[case("multipart/form-data", "multipart/form-data")]
#[case("multipart/form-data;boundary=X", "multipart/form-data")]
#[case("multipart/form-data boundary=X", "multipart/form-data")]
#[case("multipart/form-data,boundary=X", "multipart/form-data")]
#[case("multipart/FoRm-data", "multipart/form-data")]
#[case("multipart/form-data\t boundary=X", "multipart/form-data\t")]
#[case(" \tmultipart/form-data boundary=X", "multipart/form-data")]
#[case("", "")]
fn test_parse_content_type(#[case] input: &str, #[case] expected: &str) {
assert_eq!(
parse_content_type(input.as_bytes()).unwrap(),
Bstr::from(expected)
);
}
}
rust/htp/src/request.rs
+1624
View File
File diff suppressed because it is too large Load Diff
rust/htp/src/request_generic.rs
+306
View File
@@ -0,0 +1,306 @@
use crate::{
bstr::Bstr,
config::HtpUnwanted,
connection_parser::ConnectionParser,
error::Result,
headers::Flags as HeaderFlags,
parsers::{parse_content_length, parse_protocol},
request::HtpMethod,
transaction::{Header, HtpProtocol},
util::{
is_space, take_ascii_whitespace, take_is_space, take_not_is_space, take_until_null,
FlagOperations, HtpFlags,
},
};
use nom::{bytes::complete::take_while, error::ErrorKind, sequence::tuple};
use std::cmp::Ordering;
impl ConnectionParser {
/// Extract one request header. A header can span multiple lines, in
/// which case they will be folded into one before parsing is attempted.
fn process_request_header_generic(&mut self, header: Header) -> Result<()> {
// Try to parse the header.
let mut repeated = false;
let reps = self.request().request_header_repetitions;
let mut update_reps = false;
// Do we already have a header with the same name?
if let Some((_, h_existing)) = self
.request_mut()
.request_headers
.get_nocase_mut(header.name.as_slice())
{
// TODO Do we want to have a list of the headers that are
// allowed to be combined in this way?
if !h_existing.flags.is_set(HtpFlags::FIELD_REPEATED) {
// This is the second occurence for this header.
repeated = true;
} else if reps < 64 {
update_reps = true;
} else {
return Ok(());
}
// For simplicity reasons, we count the repetitions of all headers
// Keep track of repeated same-name headers.
h_existing.flags.set(HtpFlags::FIELD_REPEATED);
// Having multiple C-L headers is against the RFC but
// servers may ignore the subsequent headers if the values are the same.
if header.name.cmp_nocase("Content-Length") == Ordering::Equal {
// Don't use string comparison here because we want to
// ignore small formatting differences.
let existing_cl = parse_content_length(&h_existing.value, None);
let new_cl = parse_content_length(&header.value, None);
// Ambiguous response C-L value.
if existing_cl.is_none() || new_cl.is_none() || existing_cl != new_cl {
htp_warn!(
self.logger,
HtpLogCode::DUPLICATE_CONTENT_LENGTH_FIELD_IN_REQUEST,
"Ambiguous request C-L value"
);
}
} else {
// Add to the existing header.
h_existing.value.extend_from_slice(b", ");
h_existing.value.extend_from_slice(header.value.as_slice());
}
} else {
self.request_mut()
.request_headers
.add(header.name.clone(), header);
}
if update_reps {
self.request_mut().request_header_repetitions =
self.request().request_header_repetitions.wrapping_add(1)
}
if repeated {
htp_warn!(
self.logger,
HtpLogCode::REQUEST_HEADER_REPETITION,
"Repetition for header"
);
}
Ok(())
}
/// Generic request header parser.
pub(crate) fn process_request_headers_generic<'a>(
&mut self,
data: &'a [u8],
) -> Result<(&'a [u8], bool)> {
let rc = self.request_mut().request_header_parser.headers()(data);
if let Ok((remaining, (headers, eoh))) = rc {
for h in headers {
let mut flags = 0;
let name_flags = h.name.flags;
// Ignore LWS after field-name.
if name_flags.is_set(HeaderFlags::NAME_TRAILING_WHITESPACE) {
// Log only once per transaction.
htp_warn_once!(
self.logger,
HtpLogCode::REQUEST_INVALID_LWS_AFTER_NAME,
"Request field invalid: LWS after name",
self.request_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
//If name has leading whitespace, probably invalid folding
if name_flags.is_set(HeaderFlags::NAME_LEADING_WHITESPACE) {
// Invalid folding.
// Warn only once per transaction.
htp_warn_once!(
self.logger,
HtpLogCode::Invalid_REQUEST_FIELD_FOLDING,
"Invalid request field folding",
self.request_mut().flags,
flags,
HtpFlags::Invalid_FOLDING
);
}
// Check that field-name is a token
if name_flags.is_set(HeaderFlags::NAME_NON_TOKEN_CHARS) {
// Incorrectly formed header name.
// Log only once per transaction.
htp_warn_once!(
self.logger,
HtpLogCode::REQUEST_HEADER_INVALID,
"Request header name is not a token",
self.request_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
// No colon?
if name_flags.is_set(HeaderFlags::MISSING_COLON) {
// Log only once per transaction.
// We handle this case as a header with an empty name, with the value equal
// to the entire input string.
// TODO Apache will respond to this problem with a 400.
// Now extract the name and the value
htp_warn_once!(
self.logger,
HtpLogCode::REQUEST_FIELD_MISSING_COLON,
"Request field invalid: colon missing",
self.request_mut().flags,
flags,
HtpFlags::FIELD_UNPARSEABLE
);
} else if name_flags.is_set(HeaderFlags::NAME_EMPTY) {
// Empty header name.
// Log only once per transaction.
htp_warn_once!(
self.logger,
HtpLogCode::REQUEST_INVALID_EMPTY_NAME,
"Request field invalid: empty name",
self.request_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
self.process_request_header_generic(Header::new_with_flags(
h.name.name.into(),
h.value.value.into(),
flags,
))?;
}
Ok((remaining, eoh))
} else {
Ok((data, false))
}
}
/// Parses a single request line.
pub(crate) fn parse_request_line_generic_ex(
&mut self,
request_line: &[u8],
nul_terminates: bool,
) -> Result<()> {
let mut mstart: bool = false;
let mut data: &[u8] = request_line;
if nul_terminates {
if let Ok((_, before_null)) = take_until_null(data) {
data = before_null
}
}
// The request method starts at the beginning of the
// line and ends with the first whitespace character.
let method_parser = tuple::<_, _, (_, ErrorKind), _>
// skip past leading whitespace. IIS allows this
((take_is_space,
take_not_is_space,
// Ignore whitespace after request method. The RFC allows
// for only one SP, but then suggests any number of SP and HT
// should be permitted. Apache uses isspace(), which is even
// more permitting, so that's what we use here.
take_ascii_whitespace()
));
if let Ok((remaining, (ls, method, ws))) = method_parser(data) {
if !ls.is_empty() {
htp_warn!(
self.logger,
HtpLogCode::REQUEST_LINE_LEADING_WHITESPACE,
"Request line: leading whitespace"
);
let requestline_leading_whitespace_unwanted =
self.cfg.requestline_leading_whitespace_unwanted;
if requestline_leading_whitespace_unwanted != HtpUnwanted::Ignore {
// reset mstart so that we copy the whitespace into the method
mstart = true;
// set expected response code to this anomaly
self.request_mut().response_status_expected_number =
requestline_leading_whitespace_unwanted
}
}
if mstart {
self.request_mut().request_method =
Some(Bstr::from([&ls[..], &method[..]].concat()));
} else {
self.request_mut().request_method = Some(Bstr::from(method));
}
if let Some(request_method) = &self.request().request_method {
self.request_mut().request_method_number =
HtpMethod::new(request_method.as_slice());
}
// Too much performance overhead for fuzzing
if ws.iter().any(|&c| c != 0x20) {
htp_warn!(
self.logger,
HtpLogCode::METHOD_DELIM_NON_COMPLIANT,
"Request line: non-compliant delimiter between Method and URI"
);
}
if remaining.is_empty() {
// No, this looks like a HTTP/0.9 request.
self.request_mut().is_protocol_0_9 = true;
self.request_mut().request_protocol_number = HtpProtocol::V0_9;
if self.request().request_method_number == HtpMethod::Unknown {
htp_warn!(
self.logger,
HtpLogCode::REQUEST_LINE_UNKNOWN_METHOD,
"Request line: unknown method only"
);
}
return Ok(());
}
let uri_protocol_parser = tuple::<_, _, (_, ErrorKind), _>
// The URI ends with the first whitespace.
((take_while(|c: u8| c != 0x20),
// Ignore whitespace after URI.
take_is_space)
);
if let Ok((mut protocol, (mut uri, _))) = uri_protocol_parser(remaining) {
if uri.len() == remaining.len() && uri.iter().any(|&c| is_space(c)) {
// warn regardless if we've seen non-compliant chars
htp_warn!(
self.logger,
HtpLogCode::URI_DELIM_NON_COMPLIANT,
"Request line: URI contains non-compliant delimiter"
);
// if we've seen some 'bad' delimiters, we retry with those
let uri_protocol_parser2 =
tuple::<_, _, (_, ErrorKind), _>((take_not_is_space, take_is_space));
if let Ok((protocol2, (uri2, _))) = uri_protocol_parser2(remaining) {
uri = uri2;
protocol = protocol2;
}
}
self.request_mut().request_uri = Some(Bstr::from(uri));
// Is there protocol information available?
if protocol.is_empty() {
// No, this looks like a HTTP/0.9 request.
self.request_mut().is_protocol_0_9 = true;
self.request_mut().request_protocol_number = HtpProtocol::V0_9;
if self.request().request_method_number == HtpMethod::Unknown {
htp_warn!(
self.logger,
HtpLogCode::REQUEST_LINE_UNKNOWN_METHOD_NO_PROTOCOL,
"Request line: unknown method and no protocol"
);
}
return Ok(());
}
// The protocol information continues until the end of the line.
self.request_mut().request_protocol = Some(Bstr::from(protocol));
self.request_mut().request_protocol_number =
parse_protocol(protocol, &mut self.logger);
if self.request().request_method_number == HtpMethod::Unknown
&& self.request().request_protocol_number == HtpProtocol::Invalid
{
htp_warn!(
self.logger,
HtpLogCode::REQUEST_LINE_UNKNOWN_METHOD_INVALID_PROTOCOL,
"Request line: unknown method and invalid protocol"
);
}
}
}
Ok(())
}
}
rust/htp/src/response.rs
+1602
View File
File diff suppressed because it is too large Load Diff
rust/htp/src/response_generic.rs
+220
View File
@@ -0,0 +1,220 @@
use crate::{
bstr::Bstr,
connection_parser::ConnectionParser,
error::Result,
headers::Flags as HeaderFlags,
parsers::{parse_content_length, parse_protocol, parse_status},
transaction::{Header, HtpProtocol, HtpResponseNumber},
util::{
take_ascii_whitespace, take_is_space, take_is_space_or_null, take_not_is_space,
FlagOperations, HtpFlags,
},
HtpStatus,
};
use nom::{error::ErrorKind, sequence::tuple};
use std::cmp::Ordering;
impl ConnectionParser {
/// Generic response line parser.
pub(crate) fn parse_response_line_generic(&mut self, response_line: &[u8]) -> Result<()> {
let response_tx = self.response_mut();
response_tx.response_protocol_number = HtpProtocol::Invalid;
response_tx.response_status = None;
response_tx.response_status_number = HtpResponseNumber::Invalid;
response_tx.response_message = None;
let response_line_parser = tuple::<_, _, (_, ErrorKind), _>((
take_is_space_or_null,
take_not_is_space,
take_is_space,
take_not_is_space,
take_ascii_whitespace(),
));
if let Ok((message, (_ls, response_protocol, ws1, status_code, ws2))) =
response_line_parser(response_line)
{
if response_protocol.is_empty() {
return Ok(());
}
response_tx.response_protocol = Some(Bstr::from(response_protocol));
self.response_mut().response_protocol_number =
parse_protocol(response_protocol, &mut self.logger);
if ws1.is_empty() || status_code.is_empty() {
return Ok(());
}
let response_tx = self.response_mut();
response_tx.response_status = Some(Bstr::from(status_code));
response_tx.response_status_number = parse_status(status_code);
if ws2.is_empty() {
return Ok(());
}
response_tx.response_message = Some(Bstr::from(message));
} else {
return Err(HtpStatus::ERROR);
}
Ok(())
}
/// Generic response header parser.
///
///Returns a tuple of the unparsed data and a boolean indicating if the EOH was seen.
pub(crate) fn process_response_headers_generic<'a>(
&mut self,
data: &'a [u8],
) -> Result<(&'a [u8], bool)> {
let rc = self.response_mut().response_header_parser.headers()(data);
if let Ok((remaining, (headers, eoh))) = rc {
for h in headers {
let mut flags = 0;
let name_flags = &h.name.flags;
let value_flags = &h.value.flags;
if value_flags.is_set(HeaderFlags::DEFORMED_EOL)
|| name_flags.is_set(HeaderFlags::DEFORMED_EOL)
{
htp_warn!(
self.logger,
HtpLogCode::DEFORMED_EOL,
"Weird response end of lines mix"
);
}
// Ignore LWS after field-name.
if name_flags.is_set(HeaderFlags::NAME_TRAILING_WHITESPACE) {
htp_warn_once!(
self.logger,
HtpLogCode::RESPONSE_INVALID_LWS_AFTER_NAME,
"Request field invalid: LWS after name",
self.response_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
//If there was leading whitespace, probably was invalid folding.
if name_flags.is_set(HeaderFlags::NAME_LEADING_WHITESPACE) {
htp_warn_once!(
self.logger,
HtpLogCode::Invalid_RESPONSE_FIELD_FOLDING,
"Invalid response field folding",
self.response_mut().flags,
flags,
HtpFlags::Invalid_FOLDING
);
flags.set(HtpFlags::FIELD_INVALID);
}
// Check that field-name is a token
if name_flags.is_set(HeaderFlags::NAME_NON_TOKEN_CHARS) {
// Incorrectly formed header name.
htp_warn_once!(
self.logger,
HtpLogCode::RESPONSE_HEADER_NAME_NOT_TOKEN,
"Response header name is not a token",
self.response_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
// No colon?
if name_flags.is_set(HeaderFlags::MISSING_COLON) {
// We handle this case as a header with an empty name, with the value equal
// to the entire input string.
// TODO Apache will respond to this problem with a 400.
// Now extract the name and the value
htp_warn_once!(
self.logger,
HtpLogCode::RESPONSE_FIELD_MISSING_COLON,
"Response field invalid: colon missing",
self.response_mut().flags,
flags,
HtpFlags::FIELD_UNPARSEABLE
);
flags.set(HtpFlags::FIELD_INVALID);
} else if name_flags.is_set(HeaderFlags::NAME_EMPTY) {
// Empty header name.
htp_warn_once!(
self.logger,
HtpLogCode::RESPONSE_INVALID_EMPTY_NAME,
"Response field invalid: empty name",
self.response_mut().flags,
flags,
HtpFlags::FIELD_INVALID
);
}
self.process_response_header_generic(Header::new_with_flags(
h.name.name.into(),
h.value.value.into(),
flags,
))?;
}
Ok((remaining, eoh))
} else {
Ok((data, false))
}
}
/// Generic response header line(s) processor, which assembles folded lines
/// into a single buffer before invoking the parsing function.
fn process_response_header_generic(&mut self, header: Header) -> Result<()> {
let mut repeated = false;
let reps = self.response().response_header_repetitions;
let mut update_reps = false;
// Do we already have a header with the same name?
if let Some((_, h_existing)) = self
.response_mut()
.response_headers
.get_nocase_mut(header.name.as_slice())
{
// Keep track of repeated same-name headers.
if !h_existing.flags.is_set(HtpFlags::FIELD_REPEATED) {
// This is the second occurence for this header.
repeated = true;
} else if reps < 64 {
update_reps = true;
} else {
return Ok(());
}
h_existing.flags.set(HtpFlags::FIELD_REPEATED);
// For simplicity reasons, we count the repetitions of all headers
// Having multiple C-L headers is against the RFC but many
// browsers ignore the subsequent headers if the values are the same.
if header.name.cmp_nocase("Content-Length") == Ordering::Equal {
// Don't use string comparison here because we want to
// ignore small formatting differences.
let existing_cl = parse_content_length(&h_existing.value, None);
let new_cl = parse_content_length(&(header.value), None);
if existing_cl.is_none() || new_cl.is_none() || existing_cl != new_cl {
// Ambiguous response C-L value.
htp_warn!(
self.logger,
HtpLogCode::DUPLICATE_CONTENT_LENGTH_FIELD_IN_RESPONSE,
"Ambiguous response C-L value"
);
}
} else {
// Add to the existing header.
h_existing.value.extend_from_slice(b", ");
h_existing.value.extend_from_slice(header.value.as_slice());
}
} else {
self.response_mut()
.response_headers
.add(header.name.clone(), header);
}
if update_reps {
self.response_mut().response_header_repetitions =
self.response().response_header_repetitions.wrapping_add(1)
}
if repeated {
htp_warn!(
self.logger,
HtpLogCode::RESPONSE_HEADER_REPETITION,
"Repetition for header"
);
}
Ok(())
}
}
rust/htp/src/table.rs
+192
View File
@@ -0,0 +1,192 @@
use crate::bstr::Bstr;
use std::ops::Index;
#[cfg(test)]
use std::{cmp::Ordering, iter::Iterator, slice::SliceIndex};
/// The table structure for key value pairs.
#[derive(Clone, Debug)]
pub(crate) struct Table<T> {
/// Entries in the table.
elements: Vec<(Bstr, T)>,
}
impl<T> Index<usize> for Table<T> {
type Output = (Bstr, T);
fn index(&self, idx: usize) -> &(Bstr, T) {
&self.elements[idx]
}
}
impl<'a, T> IntoIterator for &'a Table<T> {
type Item = &'a (Bstr, T);
type IntoIter = std::slice::Iter<'a, (Bstr, T)>;
fn into_iter(self) -> std::slice::Iter<'a, (Bstr, T)> {
self.elements.iter()
}
}
impl<'a, T> IntoIterator for &'a mut Table<T> {
type Item = &'a mut (Bstr, T);
type IntoIter = std::slice::IterMut<'a, (Bstr, T)>;
fn into_iter(self) -> std::slice::IterMut<'a, (Bstr, T)> {
self.elements.iter_mut()
}
}
impl<T> IntoIterator for Table<T> {
type Item = (Bstr, T);
type IntoIter = std::vec::IntoIter<(Bstr, T)>;
fn into_iter(self) -> std::vec::IntoIter<(Bstr, T)> {
self.elements.into_iter()
}
}
impl<T> Table<T> {
#[cfg(test)]
/// Make a new owned Table with given capacity
pub(crate) fn with_capacity(size: usize) -> Self {
Self {
elements: Vec::with_capacity(size),
}
}
#[cfg(test)]
/// Add a new tuple (key, item) to the table
pub(crate) fn add(&mut self, key: Bstr, item: T) {
self.elements.push((key, item));
}
/// Retrieve an element from a specific index.
#[cfg(test)]
pub(crate) fn get<I>(&self, index: I) -> Option<&I::Output>
where
I: SliceIndex<[(Bstr, T)]>,
{
self.elements.get(index)
}
/// Retrieve a mutable reference to an element from a specific index.
#[cfg(test)]
pub(crate) fn get_mut<I>(&mut self, index: I) -> Option<&mut I::Output>
where
I: SliceIndex<[(Bstr, T)]>,
{
self.elements.get_mut(index)
}
/// Search the table for the first tuple with a key matching the given slice, ingnoring ascii case in self
///
/// Returns None if no match is found.
#[cfg(test)]
pub(crate) fn get_nocase<K: AsRef<[u8]>>(&self, key: K) -> Option<&(Bstr, T)> {
self.elements.iter().find(|x| x.0.cmp_nocase(key.as_ref()))
}
#[cfg(test)]
/// Returns the number of elements in the table
pub(crate) fn size(&self) -> usize {
self.elements.len()
}
}
// Tests
#[test]
fn Add() {
let mut t = Table::with_capacity(1);
let mut k = Bstr::from("Key");
assert_eq!(0, t.size());
t.add(k, "Value1");
assert_eq!(1, t.size());
k = Bstr::from("AnotherKey");
t.add(k, "Value2");
assert_eq!(2, t.size());
}
#[test]
fn GetNoCase() {
let mut t = Table::with_capacity(2);
let mut k = Bstr::from("Key1");
t.add(k, "Value1");
k = Bstr::from("KeY2");
t.add(k, "Value2");
let mut result = t.get_nocase("KEY1");
let mut res = result.unwrap();
assert_eq!(Ordering::Equal, res.0.cmp_slice("Key1"));
assert_eq!("Value1", res.1);
result = t.get_nocase("keY1");
res = result.unwrap();
assert_eq!(Ordering::Equal, res.0.cmp_slice("Key1"));
assert_eq!("Value1", res.1);
result = t.get_nocase("key2");
res = result.unwrap();
assert_eq!(Ordering::Equal, res.0.cmp_slice("KeY2"));
assert_eq!("Value2", res.1);
result = t.get_nocase("NotAKey");
assert!(result.is_none());
}
#[test]
fn IndexAccess() {
let mut t = Table::with_capacity(2);
let mut k = Bstr::from("Key1");
t.add(k, "Value1");
k = Bstr::from("KeY2");
t.add(k, "Value2");
let res = &t[1];
assert_eq!(Ordering::Equal, res.0.cmp_slice("KeY2"));
assert_eq!("Value2", res.1);
assert_eq!("Value2", t.get(1).unwrap().1);
let res_mut = t.get_mut(1).unwrap();
res_mut.1 = "Value3";
assert_eq!("Value3", t.get(1).unwrap().1);
}
#[test]
fn Iterators() {
let mut table = Table::with_capacity(2);
table.add("1".into(), "abc".to_string());
table.add("2".into(), "def".to_string());
let mut iter_ref: std::slice::Iter<(Bstr, String)> = (&table).into_iter();
let (key1, _): &(Bstr, String) = iter_ref.next().unwrap();
assert_eq!(key1, &"1");
assert_eq!(table.get_nocase("1").unwrap().1, "abc");
let mut iter_mut_ref: std::slice::IterMut<(Bstr, String)> = (&mut table).into_iter();
let (key1, ref mut val1): &mut (Bstr, String) = iter_mut_ref.next().unwrap();
*val1 = "xyz".to_string();
assert_eq!(key1, &"1");
assert_eq!(table.get_nocase("1").unwrap().1, "xyz");
let mut iter_owned: std::vec::IntoIter<(Bstr, String)> = table.into_iter();
let (key1, val1) = iter_owned.next().unwrap();
assert_eq!(key1, "1");
assert_eq!(val1, "xyz");
}
#[test]
fn Table_Misc() {
let mut t: Table<&str> = Table::with_capacity(2);
let mut pkey = Bstr::with_capacity(1);
pkey.add("p");
let mut qkey = Bstr::with_capacity(1);
qkey.add("q");
t.add(pkey, "1");
t.add(qkey, "2");
assert!(t.get_nocase("z").is_none());
assert_eq!("1", t.get_nocase("p").unwrap().1);
}
rust/htp/src/test/common.rs
+598
View File
@@ -0,0 +1,598 @@
#![allow(non_snake_case)]
use crate::{
bstr::Bstr,
config::{Config, HtpServerPersonality},
connection_parser::{ConnectionParser, HtpStreamState, ParserData},
error::Result,
transaction::Transaction,
};
use std::{
env,
iter::IntoIterator,
net::{IpAddr, Ipv4Addr},
path::PathBuf,
time::SystemTime,
};
use time::OffsetDateTime;
#[derive(Debug)]
enum Chunk {
Client(ParserData<'static>),
Server(ParserData<'static>),
}
/// A structure to hold callback data
pub(super) struct MainUserData {
/// Call order of callbacks
pub order: Vec<String>,
/// Request data from callbacks
pub request_data: Vec<Bstr>,
/// Response data from callbacks
pub response_data: Vec<Bstr>,
}
impl Default for MainUserData {
/// Make a new user data struct
fn default() -> Self {
Self {
order: Vec::new(),
request_data: Vec::with_capacity(5),
response_data: Vec::with_capacity(5),
}
}
}
#[derive(Debug)]
struct TestInput {
chunks: Vec<Chunk>,
}
impl IntoIterator for TestInput {
type Item = Chunk;
type IntoIter = std::vec::IntoIter<Self::Item>;
fn into_iter(self) -> Self::IntoIter {
self.chunks.into_iter()
}
}
impl From<PathBuf> for TestInput {
fn from(file: PathBuf) -> Self {
let input = std::fs::read(file)
.expect("Could not read file {:?}. Do you need to set a base dir in env('srcdir')?");
TestInput::from(input.as_slice())
}
}
impl From<&[u8]> for TestInput {
fn from(input: &[u8]) -> Self {
let mut test_input = TestInput { chunks: Vec::new() };
let mut current = Vec::<u8>::new();
let mut client = true;
let mut is_gap = false;
let mut start = true;
for line in input.split_inclusive(|c| *c == b'\n') {
if line.len() >= 4
&& line.len() <= 5
&& (&line[0..3] == b"<<<"
|| &line[0..3] == b"<><"
|| &line[0..3] == b">>>"
|| &line[0..3] == b"><>")
&& (line.len() == 4 || line[3] == b'\r')
&& line[line.len() - 1] == b'\n'
{
if !current.is_empty() {
// Pop off the CRLF from the last line, which
// just separates the previous data from the
// boundary <<< >>> chars and isn't actual data
if let Some(b'\n') = current.last() {
current.pop();
}
if let Some(b'\r') = current.last() {
current.pop();
}
test_input.append(client, current, is_gap);
current = Vec::<u8>::new();
}
// Client represented by first char is >
client = line[0] == b'>';
// Gaps represented by <>< or ><>
is_gap = line[0] != line[1];
start = false;
} else {
if start {
// we need to start with an indicated direction
return test_input;
}
current.append(&mut line.to_vec());
}
}
test_input.append(client, current, is_gap);
test_input
}
}
impl TestInput {
fn append(&mut self, client: bool, data: Vec<u8>, is_gap: bool) {
let chunk = match (client, is_gap) {
// client gap
(true, true) => Chunk::Client(data.len().into()),
// client data
(true, false) => Chunk::Client(data.into()),
// server gap
(false, true) => Chunk::Server(data.len().into()),
// server data
(false, false) => Chunk::Server(data.into()),
};
self.chunks.push(chunk);
}
}
/// Error types
#[derive(Debug)]
pub(super) enum TestError {
/// The parser entered the Error state
StreamError,
}
/// Test harness
#[derive(Debug)]
pub(super) struct Test {
/// The connection parse
pub connp: ConnectionParser,
/// The base directory for the crate - used to find files.
pub basedir: Option<PathBuf>,
}
/// Return a default Config to use with tests
pub(super) fn TestConfig() -> Config {
let mut cfg = Config::default();
cfg.set_server_personality(HtpServerPersonality::APACHE_2)
.unwrap();
// The default bomb limit may be slow in some development environments causing tests to fail.
cfg.compression_options
.set_time_limit(10 * cfg.compression_options.get_time_limit());
cfg
}
impl Test {
/// Make a new test with the given config
pub(super) fn new(cfg: Config) -> Self {
let basedir = if let Ok(dir) = std::env::var("srcdir") {
Some(PathBuf::from(dir))
} else if let Ok(dir) = env::var("CARGO_MANIFEST_DIR") {
let mut base = PathBuf::from(dir);
base.push("src");
base.push("test");
base.push("files");
Some(base)
} else {
None
};
let cfg = Box::leak(Box::new(cfg));
let connp = ConnectionParser::new(cfg);
Test { connp, basedir }
}
/// Make a new test with the default TestConfig and register body callbacks.
pub(super) fn new_with_callbacks() -> Self {
let mut cfg = TestConfig();
cfg.register_request_start(request_start);
cfg.register_request_complete(request_complete);
cfg.register_response_start(response_start);
cfg.register_response_complete(response_complete);
cfg.register_response_body_data(response_body_data);
cfg.register_request_body_data(request_body_data);
cfg.register_transaction_complete(transaction_complete);
let mut t = Test::new(cfg);
// Configure user data and callbacks
t.connp
.response_mut()
.unwrap()
.set_user_data(Box::<MainUserData>::default());
t
}
/// Open a connection on the underlying ConnectionParser. Useful if you
/// want to send data directly to the ConnectionParser after.
pub(super) fn open_connection(&mut self, tv_start: Option<OffsetDateTime>) {
self.connp.open(
Some(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1))),
Some(10000),
Some(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1))),
Some(80),
tv_start,
);
}
fn run(&mut self, test: TestInput) -> std::result::Result<(), TestError> {
let tv_start = Some(OffsetDateTime::from(SystemTime::now()));
self.open_connection(tv_start);
let mut request_buf: Option<ParserData> = None;
let mut response_buf: Option<ParserData> = None;
for chunk in test {
match chunk {
Chunk::Client(data) => {
let rc = self.connp.request_data(data.clone(), tv_start);
if rc == HtpStreamState::ERROR {
return Err(TestError::StreamError);
}
if rc == HtpStreamState::DATA_OTHER {
let consumed = self.connp.request_data_consumed();
let remaining = data.clone().into_owned();
remaining.consume(consumed);
request_buf = Some(remaining);
}
}
Chunk::Server(data) => {
// If we have leftover data from before then use it first
if let Some(response_remaining) = response_buf {
let rc = self
.connp
.response_data(response_remaining.as_slice().into(), tv_start);
response_buf = None;
if rc == HtpStreamState::ERROR {
return Err(TestError::StreamError);
}
}
// Now use up this data chunk
let rc = self.connp.response_data(data.clone(), tv_start);
if rc == HtpStreamState::ERROR {
return Err(TestError::StreamError);
}
if rc == HtpStreamState::DATA_OTHER {
let consumed = self.connp.response_data_consumed();
let remaining = data.clone().into_owned();
remaining.consume(consumed);
response_buf = Some(remaining);
}
// And check if we also had some input data buffered
if let Some(request_remaining) = request_buf {
let rc = self
.connp
.request_data(request_remaining.as_slice().into(), tv_start);
request_buf = None;
if rc == HtpStreamState::ERROR {
return Err(TestError::StreamError);
}
}
}
}
}
// Clean up any remaining server data
if let Some(response_remaining) = response_buf {
let rc = self
.connp
.response_data(response_remaining.as_slice().into(), tv_start);
if rc == HtpStreamState::ERROR {
return Err(TestError::StreamError);
}
}
self.connp
.close(Some(OffsetDateTime::from(SystemTime::now())));
Ok(())
}
/// Run on a slice of input data. Used with fuzzing.
pub(super) fn run_slice(&mut self, slice: &[u8]) -> std::result::Result<(), TestError> {
self.run(TestInput::from(slice))
}
/// Run on a file path. Used in integration tests.
pub(super) fn run_file(&mut self, file: &str) -> std::result::Result<(), TestError> {
let testfile = if let Some(base) = &self.basedir {
let mut path = base.clone();
path.push(file);
path
} else {
PathBuf::from(file)
};
self.run(TestInput::from(testfile))
}
}
fn request_start(tx: &mut Transaction) -> Result<()> {
let id = tx.index;
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
user_data.order.push(format!("request_start {}", id));
Ok(())
}
fn request_complete(tx: &mut Transaction) -> Result<()> {
let id = tx.index;
let user_data = &mut tx.user_data_mut::<MainUserData>().unwrap();
user_data.order.push(format!("request_complete {}", id));
Ok(())
}
fn response_start(tx: &mut Transaction) -> Result<()> {
let id = tx.index;
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
user_data.order.push(format!("response_start {}", id));
Ok(())
}
fn response_complete(tx: &mut Transaction) -> Result<()> {
let id = tx.index;
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
user_data.order.push(format!("response_complete {}", id));
Ok(())
}
fn transaction_complete(tx: &mut Transaction) -> Result<()> {
let id = tx.index;
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
user_data.order.push(format!("transaction_complete {}", id));
Ok(())
}
fn response_body_data(tx: &mut Transaction, d: &ParserData) -> Result<()> {
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
let bstr = if d.is_gap() {
Bstr::with_capacity(d.len())
} else {
Bstr::from(d.as_slice())
};
user_data.response_data.push(bstr);
Ok(())
}
fn request_body_data(tx: &mut Transaction, d: &ParserData) -> Result<()> {
let user_data = tx.user_data_mut::<MainUserData>().unwrap();
let bstr = if d.is_gap() {
Bstr::with_capacity(d.len())
} else {
Bstr::from(d.as_slice())
};
user_data.request_data.push(bstr);
Ok(())
}
#[no_mangle]
/// Creates a Fuzz test runner, and runs a byte slice on it
/// # Safety
/// Input pointer must be non-null.
pub unsafe extern "C" fn libhtprsFuzzRun(
input: *const u8, input_len: u32,
) -> *mut std::os::raw::c_void {
let mut cfg = TestConfig();
cfg.set_server_personality(HtpServerPersonality::IDS)
.unwrap();
let mut t = Test::new(cfg);
let data = std::slice::from_raw_parts(input, input_len as usize);
t.run_slice(data).ok();
let boxed = Box::new(t);
Box::into_raw(boxed) as *mut _
}
#[no_mangle]
/// Frees a Fuzz test runner
/// # Safety
/// Input pointer must be non-null.
pub unsafe extern "C" fn libhtprsFreeFuzzRun(state: *mut std::os::raw::c_void) {
//just unbox
std::mem::drop(Box::from_raw(state as *mut Test));
}
#[no_mangle]
/// Gets connection parser out of a test runner
/// # Safety
/// Input pointer must be non-null.
pub unsafe extern "C" fn libhtprsFuzzConnp(t: *mut std::os::raw::c_void) -> *mut ConnectionParser {
let state = t as *mut Test;
&mut (*state).connp
}
#[macro_export]
/// Cstring converter
macro_rules! cstr {
( $x:expr ) => {{
CString::new($x).unwrap().as_ptr()
}};
}
/// Compares a transaction's header value to an expected value.
///
/// The `attr` argument is meant to be either `request_headers` or `response_headers`.
///
/// Example usage:
/// assert_header_eq!(tx, request_headers, "host", ""www.example.com");
macro_rules! assert_header_eq {
($tx:expr, $attr:ident, $key:expr, $val:expr) => {{
let header = &(*$tx).$attr
.get_nocase_nozero($key)
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref());
assert_eq!(*header.value, $val);
}};
($tx:expr, $attr:ident, $key:expr, $val:expr,) => {{
assert_header_eq!($tx, $attr, $key, $val);
}};
($tx:expr, $attr:ident, $key:expr, $val:expr, $($arg:tt)+) => {{
let header = (*(*$tx).$attr)
.get_nocase_nozero($key)
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref())
.1
.as_ref()
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref());
assert_eq!(*header.value, $val, $($arg)*);
}};
}
pub(crate) use assert_header_eq;
/// Compares a transaction's request header value to an expected value.
///
/// Example usage:
/// assert_request_header_eq!(tx, "host", ""www.example.com");
macro_rules! assert_request_header_eq {
($tx:expr, $key:expr, $val:expr) => {{
assert_header_eq!($tx, request_headers, $key, $val);
}};
($tx:expr, $key:expr, $val:expr,) => {{
assert_header_eq!($tx, request_headers, $key, $val);
}};
($tx:expr, $key:expr, $val:expr, $($arg:tt)+) => {{
assert_header_eq!($tx, request_headers, $val, $($arg)*);
}};
}
pub(crate) use assert_request_header_eq;
/// Compares a transaction's response header value to an expected value.
///
/// Example usage:
/// assert_response_header_eq!(tx, "content-encoding", ""gzip");
macro_rules! assert_response_header_eq {
($tx:expr, $key:expr, $val:expr) => {{
assert_header_eq!($tx, response_headers, $key, $val);
}};
($tx:expr, $key:expr, $val:expr,) => {{
assert_header_eq!($tx, response_headers, $key, $val);
}};
($tx:expr, $key:expr, $val:expr, $($arg:tt)+) => {{
assert_header_eq!($tx, response_headers, $val, $($arg)*);
}};
}
pub(crate) use assert_response_header_eq;
/// Asserts that a transaction's response contains a flag.
///
/// Example usage:
/// assert_response_header_flag_contains!(tx, "Content-Length", Flags::FIELD_REPEATED);
macro_rules! assert_response_header_flag_contains {
($tx:expr, $key:expr, $val:expr) => {{
let header = &(*$tx).response_headers
.get_nocase_nozero($key)
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref());
assert!(header.flags.is_set($val));
}};
($tx:expr, $key:expr, $val:expr,) => {{
assert_response_header_flag_contains!($tx, response_headers, $key, $val);
}};
($tx:expr, $key:expr, $val:expr, $($arg:tt)+) => {{
let header = (*(*$tx).response_headers)
.get_nocase_nozero($key)
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref())
.1
.as_ref()
.expect(format!(
"expected header '{}' to exist at {}:{}:{}",
$key,
file!(),
line!(),
column!()
).as_ref());
assert_eq!(*header.value, $val, $($arg)*);
assert!((*header).flags.is_set($val), $($arg)*);
}};
}
pub(crate) use assert_response_header_flag_contains;
/// Assert the common evader request values are as expected
///
/// Example usage:
/// assert_evader_request!(tx, "url");
macro_rules! assert_evader_request {
($tx:expr, $url:expr) => {{
assert!(($tx).request_method.as_ref().unwrap().eq_slice("GET"));
assert!(($tx).request_uri.as_ref().unwrap().eq_slice($url));
assert_eq!(HtpProtocol::V1_1, ($tx).request_protocol_number);
assert_header_eq!($tx, request_headers, "host", "evader.example.com");
}};
}
pub(crate) use assert_evader_request;
/// Assert the common evader response values are as expected
///
/// Example usage:
/// assert_evader_response!(tx);
macro_rules! assert_evader_response {
($tx:expr) => {{
assert_eq!(HtpProtocol::V1_1, ($tx).response_protocol_number);
assert!(($tx).response_status_number.eq_num(200));
assert_response_header_eq!($tx, "Content-type", "application/octet-stream");
assert_response_header_eq!(
$tx,
"Content-disposition",
"attachment; filename=\"eicar.txt\""
);
assert!(($tx)
.response_headers
.get_nocase_nozero("Connection")
.is_some());
}};
}
pub(crate) use assert_evader_response;
/// Assert the response transfer encoding is detected as chunked
///
/// Example usage:
/// assert_evader_chunked_response!(tx);
macro_rules! assert_evader_chunked {
($tx:expr) => {{
assert_eq!($tx.response_transfer_coding, HtpTransferCoding::Chunked);
assert_response_header_eq!($tx, "Yet-Another-Header", "foo");
assert_eq!(68, ($tx).response_entity_len);
assert_eq!(156, ($tx).response_message_len);
let user_data = ($tx).user_data::<MainUserData>().unwrap();
assert!(user_data.request_data.is_empty());
assert_eq!(17, user_data.response_data.len());
assert_eq!(b"X5O!".as_ref(), (&user_data.response_data[0]).as_slice());
assert_eq!(b"P%@A".as_ref(), (&user_data.response_data[1]).as_slice());
assert_eq!(b"P[4\\".as_ref(), (&user_data.response_data[2]).as_slice());
assert_eq!(b"PZX5".as_ref(), (&user_data.response_data[3]).as_slice());
assert_eq!(b"4(P^".as_ref(), (&user_data.response_data[4]).as_slice());
assert_eq!(b")7CC".as_ref(), (&user_data.response_data[5]).as_slice());
assert_eq!(b")7}$".as_ref(), (&user_data.response_data[6]).as_slice());
assert_eq!(b"EICA".as_ref(), (&user_data.response_data[7]).as_slice());
assert_eq!(b"R-ST".as_ref(), (&user_data.response_data[8]).as_slice());
assert_eq!(b"ANDA".as_ref(), (&user_data.response_data[9]).as_slice());
assert_eq!(b"RD-A".as_ref(), (&user_data.response_data[10]).as_slice());
assert_eq!(b"NTIV".as_ref(), (&user_data.response_data[11]).as_slice());
assert_eq!(b"IRUS".as_ref(), (&user_data.response_data[12]).as_slice());
assert_eq!(b"-TES".as_ref(), (&user_data.response_data[13]).as_slice());
assert_eq!(b"T-FI".as_ref(), (&user_data.response_data[14]).as_slice());
assert_eq!(b"LE!$".as_ref(), (&user_data.response_data[15]).as_slice());
assert_eq!(b"H+H*".as_ref(), (&user_data.response_data[16]).as_slice());
assert_eq!(HtpRequestProgress::COMPLETE, ($tx).request_progress);
assert_eq!(HtpResponseProgress::COMPLETE, ($tx).response_progress);
}};
}
pub(crate) use assert_evader_chunked;
rust/htp/src/test/files/00-adhoc.t
+14
View File
@@ -0,0 +1,14 @@
>>>
GET /?p=%20 HTTP/1.0
User-Agent: Mozilla
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/01-get.t
+14
View File
@@ -0,0 +1,14 @@
>>>
GET /?p=%20 HTTP/1.0
User-Agent: Mozilla
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/02-header-test-apache2.t
BIN
View File
Binary file not shown.
rust/htp/src/test/files/03-post-urlencoded.t
+34
View File
@@ -0,0 +1,34 @@
>>>
POST /?qsp1=1&%20p%20q=2&u=Ivan+Risti%C4%87_Ivan+Risti%C4%87_Ivan+Risti%C4%87_Ivan+Risti%C4%87_Ivan+Risti%C4%87_Ivan+Risti%C4%87_ HTTP/1.0
Content-Length: 12
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla
p=0123456789
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
>>>
GET / HTTP/1.0
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Transfer-Encoding: chunked
9
012345678
1
9
0
rust/htp/src/test/files/04-post-urlencoded-chunked.t
+26
View File
@@ -0,0 +1,26 @@
>>>
POST / HTTP/1.1
Transfer-Encoding: chunked
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla
Cookie: 1
b
p=012345678
1
9
0
Cookie:
>>>
2
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/05-expect.t
+39
View File
@@ -0,0 +1,39 @@
>>>
POST / HTTP/1.1
User-Agent: curl/7.18.2 (i486-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.8 libssh2/0.18
Accept: */*
Content-Length: 216
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------07869933ca1b
<<<
HTTP/1.1 100 Continue
Header1: This
Header2: That
>>>
------------------------------07869933ca1b
Content-Disposition: form-data; name="file"; filename="404.php"
Content-Type: application/octet-stream
>>>
<? echo "404"; ?>
>>>
------------------------------07869933ca1b--
<<<
HTTP/1.1 200 OK
Date: Tue, 03 Nov 2009 09:27:47 GMT
Server: Apache
Last-Modified: Thu, 30 Apr 2009 12:20:49 GMT
ETag: "2dcada-2d-468c4b9ec6a40"
Accept-Ranges: bytes
Content-Length: 45
Vary: Accept-Encoding
Content-Type: text/html
<html><body><h1>It works!</h1></body></html>
rust/htp/src/test/files/06-uri-normal.t
+9
View File
@@ -0,0 +1,9 @@
>>>
GET http://username:password@www.example.com:8080/sub/folder/file.jsp?p=q#f HTTP/1.0
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
rust/htp/src/test/files/07-pipelined-connection.t
+15
View File
@@ -0,0 +1,15 @@
>>>
GET /first HTTP/1.1
GET /second HTTP/1.1
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
rust/htp/src/test/files/08-not-pipelined-connection.t
+18
View File
@@ -0,0 +1,18 @@
>>>
GET /first HTTP/1.1
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
>>>
GET /second HTTP/1.1
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
rust/htp/src/test/files/09-multi-packet-request-head.t
+14
View File
@@ -0,0 +1,14 @@
>>>
GET / HTTP/1.0
>>>
Host: www.example.com
>>>
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
rust/htp/src/test/files/10-host-in-headers.t
+34
View File
@@ -0,0 +1,34 @@
>>>
GET / HTTP/1.1
Host: www.example.com
<<<
HTTP/1.0 200 OK
Content-Length: 12
Hello World!
>>>
GET / HTTP/1.1
Host: www.example.com.
<<<
HTTP/1.0 200 OK
Content-Length: 12
>>>
GET / HTTP/1.1
Host: WwW.ExamPle.cOm
<<<
HTTP/1.0 200 OK
Content-Length: 12
>>>
GET / HTTP/1.1
Host: www.example.com:80
<<<
HTTP/1.0 200 OK
Content-Length: 12
rust/htp/src/test/files/100-auth-digest-escaped-quote.t
+8
View File
@@ -0,0 +1,8 @@
>>>
GET / HTTP/1.1
Host: www.example.com
Authorization: Digest username="ivan\"r\"", realm="Book Review",
nonce="OgmPjb/jAwA=7c5a49c2ed9416dba1b04b5307d6d935f74a859d",
uri="/review/", algorithm=MD5, response="3c430d26043cc306e0282635929d57cb",
qop=auth, nc=00000004, cnonce="c3bcee9534c051a0"
rust/htp/src/test/files/101-request-cookies-2.t
+16
View File
@@ -0,0 +1,16 @@
>>>
GET / HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
Cookie: =0; p=1; q=2; =; ; z=
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/102-request-cookies-3.t
+16
View File
@@ -0,0 +1,16 @@
>>>
GET / HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
Cookie: = ; a=1; b=2 ; =7; c=double=equal
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/103-request-cookies-4.t
+16
View File
@@ -0,0 +1,16 @@
>>>
GET / HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
Cookie: = ;;c=1; a=1 ; b=2;
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/104-request-cookies-5.t
+16
View File
@@ -0,0 +1,16 @@
>>>
GET / HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
Cookie:
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/105-expect-100.t
+27
View File
@@ -0,0 +1,27 @@
>>>
PUT /forbidden HTTP/1.1
Content-Length: 14
Expect: 100-continue
<<<
HTTP/1.0 401 Forbidden
Content-Length: 0
>>>
POST /ok HTTP/1.1
Content-Length: 14
Expect: 100-continue
<<<
HTTP/1.0 100 continue
Content-Length: 0
>>>
Hello People!
<<<
HTTP/1.0 200 OK
rust/htp/src/test/files/106-tunnelled-1.t
+15
View File
@@ -0,0 +1,15 @@
>>>
CONNECT abc:443 HTTP/1.1
User-Agent: Victor/1.0
GET / HTTP/1.1
User-Agent: Victor/1.0
<<<
HTTP/1.1 200 OK
Server: VictorServer/1.0
rust/htp/src/test/files/107-response_unknown_status.t
+12
View File
@@ -0,0 +1,12 @@
>>>
GET /ld/index.php?id=412784631&cid=0064&version=4&name=try HTTP/1.1
Accept: */*
User-Agent:LD-agent
Host: 209.405.196.16
<<<
NOTHTTP REALLY
SOMEOTHERDATA
STILLNOTHTTP
SOMEMOREOTHERDATA
rust/htp/src/test/files/108-response-headers-cr-only.t
+9
View File
@@ -0,0 +1,9 @@
>>>
GET /index.html HTTP/1.0
Host: www.google.org
User-Agent: Mozilla/5.0
<<<
HTTP/1.0 200 message
Content-Type: text/html␍Content-Length: 7
rust/htp/src/test/files/109-response-headers-deformed-eol.t
+12
View File
@@ -0,0 +1,12 @@
>>>
GET /index.html HTTP/1.0
Host: www.google.org
User-Agent: Mozilla/5.0
<<<
HTTP/1.0 200 message
Content-Type: text/html
Content-Length: 6
␍
abcdef
rust/htp/src/test/files/11-response-stream-closure.t
+13
View File
@@ -0,0 +1,13 @@
>>>
GET / HTTP/1.0
User-Agent: Mozilla
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Hello World!
rust/htp/src/test/files/110-response-folded-headers-2.t
+14
View File
@@ -0,0 +1,14 @@
>>>
POST / HTTP/1.0
Content-Length: 12
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla
p=0123456789
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Server
Connection: close
rust/htp/src/test/files/111-response-headers-chunked.t
+13
View File
@@ -0,0 +1,13 @@
>>>
GET /index.html HTTP/1.0
Host: www.google.org
User-Agent: Mozilla/5.0
<<<
HTTP/1.0 200 message
Content-
<<<
Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/112-response-headers-chunked-2.t
+15
View File
@@ -0,0 +1,15 @@
>>>
GET /index.html HTTP/1.0
Host: www.google.org
User-Agent: Mozilla/5.0
<<<
HTTP/1.0 200 message
Content-
<<<
Type: text/html
Content-Length:
<<<
12
Hello World!
rust/htp/src/test/files/113-response-multipart-byte-ranges.t
+23
View File
@@ -0,0 +1,23 @@
>>>
GET / HTTP/1.1
Host: www.example.com
Range: bytes=0-50, 100-150
<<<
HTTP/1.1 206 Partial content
Date: Wed, 15 Nov 1995 06:25:24 GMT
Last-modified: Wed, 15 Nov 1995 04:58:08 GMT
Content-type: multipart/byteranges; boundary=THIS_STRING_SEPARATES
--THIS_STRING_SEPARATES
Content-type: application/pdf
Content-range: bytes 500-999/8000
...the first range...
--THIS_STRING_SEPARATES
Content-type: application/pdf
Content-range: bytes 7000-7999/8000
...the second range
--THIS_STRING_SEPARATES--
rust/htp/src/test/files/114-http-2-upgrade.t
+33
View File
@@ -0,0 +1,33 @@
>>>
GET /robots.txt HTTP/1.1
Host: nghttp2.org
User-Agent: curl/7.61.0
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA
<<<
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 12
Hello World!
>>>
GET /robots.txt HTTP/1.1
Host: nghttp2.org
User-Agent: curl/7.61.0
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA
<<<
HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Upgrade: h2c
rust/htp/src/test/files/115-auth-bearer.t
+8
View File
@@ -0,0 +1,8 @@
>>>
GET /resource HTTP/1.1
Host: server.example.com
Authorization: Bearer mF_9.B5f-4.1JqM
rust/htp/src/test/files/116-request-compression.t
BIN
View File
Binary file not shown.
rust/htp/src/test/files/117-request-response-compression.t
BIN
View File
Binary file not shown.
rust/htp/src/test/files/118-post.t
+16
View File
@@ -0,0 +1,16 @@
>>>
POST / HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
Content-Length: 12
Hello World!
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hello World!
rust/htp/src/test/files/119-ambiguous-eol.t
+11
View File
@@ -0,0 +1,11 @@
>>>
POST / HTTP/1.0
␍
>>>
<<<
<<<
HTTP/1.0 200 OK
␍
<<<
rust/htp/src/test/files/12-connect-request.t
+21
View File
@@ -0,0 +1,21 @@
>>>
CONNECT www.ssllabs.com:443 HTTP/1.0
<<<
HTTP/1.1 405 Method Not Allowed
Date: Sat, 12 Dec 2009 05:08:45 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8g PHP/5.3.0
Allow: GET,HEAD,POST,OPTIONS,TRACE
Vary: Accept-Encoding
Content-Length: 230
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>405 Method Not Allowed</title>
</head><body>
<h1>Method Not Allowed</h1>
<p>The requested method CONNECT is not allowed for the URL /.</p>
</body></html>
rust/htp/src/test/files/120-request-gap.t
+43
View File
@@ -0,0 +1,43 @@
>>>
POST / HTTP/1.1
User-Agent: curl/7.18.2 (i486-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.8 libssh2/0.18
Accept: */*
Content-Length: 216
Expect: 100-continue
Content-Type: multipart/form-data; boundary=----------------------------07869933ca1b
<<<
HTTP/1.1 100 Continue
Header1: This
Header2: That
>>>
------------------------------07869933ca1b
Content-Disposition: form-data; name="file"; filename="404.php"
Content-Type: application/octet-stream
>>>
<? echo
><>
"404"
>>>
; ?>
>>>
------------------------------07869933ca1b--
<<<
HTTP/1.1 200 OK
Date: Tue, 03 Nov 2009 09:27:47 GMT
Server: Apache
Last-Modified: Thu, 30 Apr 2009 12:20:49 GMT
ETag: "2dcada-2d-468c4b9ec6a40"
Accept-Ranges: bytes
Content-Length: 45
Vary: Accept-Encoding
Content-Type: text/html
<html><body><h1>It works!</h1></body></html>
rust/htp/src/test/files/121-response-gap.t
+17
View File
@@ -0,0 +1,17 @@
>>>
GET / HTTP/1.0
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
Hell
<><
o Wo
<<<
rld!
rust/htp/src/test/files/122-response-body-data.t
+6
View File
@@ -0,0 +1,6 @@
<<<
1
2
<<<
3
4
rust/htp/src/test/files/123-response-header-bug.t
+14
View File
@@ -0,0 +1,14 @@
>>>
GET http://www.example.com:XXX/?p=%20 HTTP/1.0
Host: www.example.com
User-Agent: Mozilla
<<<
HTTP/1.0 200 OK
Date: Mon, 31 Aug 2009 20:25:50 GMT
Server: Apache
Connection: close
Content-Type: text/html
Content-Length: 12
␍Hello World!
rust/htp/src/test/files/124-response-incomplete.t
+10
View File
@@ -0,0 +1,10 @@
>>>
GET /index.html HTTP/1.0
Content-Type: text/html
Host: www.openinfosecfoundation.org
User-Agent: This is dummy message body
<<<
HTTP/1.0 200 OK
Content-Length: 7
Content-Type: text/html
rust/htp/src/test/files/13-compressed-response-gzip-ct.t
BIN
View File
Binary file not shown.
rust/htp/src/test/files/14-compressed-response-gzip-chunked.t
BIN
View File
Binary file not shown.
rust/htp/src/test/files/15-connect-complete.t
BIN
View File
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More