Compare commits

..
Author SHA1 Message Date
knakul853 507a4feaee only save what needed 2025-06-10 19:42:04 +05:30
447 changed files with 4455 additions and 18397 deletions

No files matched your search

-76
View File
@@ -1,76 +0,0 @@
# Nuclei Discussion Guidelines
## Before Creating a Discussion
1. **Search existing discussions and issues** to avoid duplicates
2. **Check the documentation** and README first
3. **Browse the FAQ** and common questions
## Bug Reports in Discussions
When reporting a bug in [Q&A Discussions](https://github.com/projectdiscovery/nuclei/discussions/categories/q-a), please include:
### Required Information:
- **Clear title** with `[BUG]` prefix (e.g., "[BUG] Nuclei crashes when...")
- **Current behavior** - What's happening now?
- **Expected behavior** - What should happen instead?
- **Steps to reproduce** - Commands or actions that trigger the issue
- **Environment details**:
- OS and version
- Nuclei version (`nuclei -version`)
- Go version (if installed via `go install`)
- **Log output** - Run with `-verbose` or `-debug` for detailed logs
- **Redact sensitive information** - Remove target URLs, credentials, etc.
### After Discussion:
- Maintainers will review and validate the bug report
- Valid bugs will be converted to issues with proper labels and tracking
- Questions and misconfigurations will be resolved in the discussion
## Feature Requests in Discussions
When requesting a feature in [Ideas Discussions](https://github.com/projectdiscovery/nuclei/discussions/categories/ideas), please include:
### Required Information:
- **Clear title** with `[FEATURE]` prefix (e.g., "[FEATURE] Add support for...")
- **Feature description** - What do you want to be added?
- **Use case** - Why is this feature needed? What problem does it solve?
- **Implementation ideas** - If you have suggestions on how it could work
- **Alternatives considered** - What other solutions have you thought about?
### After Discussion:
- Community and maintainers will discuss the feasibility
- Popular and viable features will be converted to issues
- Similar features may be grouped together
- Rejected features will be explained in the discussion
## Getting Help
For general questions, troubleshooting, and "how-to" topics:
- Use [Q&A Discussions](https://github.com/projectdiscovery/nuclei/discussions/categories/q-a)
- Join the [Discord server](https://discord.gg/projectdiscovery) #nuclei channel
- Check existing discussions for similar questions
## Discussion to Issue Conversion Process
Only maintainers can convert discussions to issues. The process:
1. **Validation** - Maintainers review the discussion for completeness and validity
2. **Classification** - Determine if it's a bug, feature, enhancement, etc.
3. **Issue creation** - Create a properly formatted issue with appropriate labels
4. **Linking** - Link the issue back to the original discussion
5. **Resolution** - Mark the discussion as resolved or close it
This process ensures:
- High-quality issues that are actionable
- Proper triage and labeling
- Reduced noise in the issue tracker
- Community involvement in the validation process
## Why This Process?
- **Better organization** - Issues contain only validated, actionable items
- **Community input** - Discussions allow for community feedback before escalation
- **Quality control** - Maintainers ensure proper formatting and information
- **Reduced maintenance** - Fewer invalid or duplicate issues to manage
- **Clear separation** - Questions vs. actual bugs/features are clearly distinguished
+6 -12
View File
@@ -36,10 +36,7 @@ body:
description: |
Steps to reproduce the behavior, for example, commands to run Nuclei.
💡 Prefer copying text output over using screenshots for easier troubleshooting.
📝 For a more detailed output that could help in troubleshooting, you may want to run Nuclei with the **`-verbose`** or **`-debug`** flags. This will provide additional insights into what's happening under the hood.<br>
📊 For performance or memory investigations, use **`-profile-mem`**, which generates `*.cpu`, `*.mem`, and `*.trace` files. Since GitHub doesn't support these formats directly, compress them (e.g., .zip or .tar.gz) and attach the archive under the "Anything else" section below.
📝 For a more detailed output that could help in troubleshooting, you may want to run Nuclei with the **`-verbose`** or **`-debug`** flags. This will provide additional insights into what's happening under the hood.
:warning: **Please redact any literal target hosts/URLs or other sensitive information.**
placeholder: |
@@ -52,10 +49,7 @@ body:
description: |
Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
💡 Prefer copying text output over using screenshots for easier troubleshooting.
📝 For a more detailed output that could help in troubleshooting, you may want to run Nuclei with the **`-verbose`** or **`-debug`** flags. This will provide additional insights into what's happening under the hood.<br>
📊 For performance or memory investigations, use **`-profile-mem`**, which generates `*.cpu`, `*.mem`, and `*.trace` files. Since GitHub doesn't support these formats directly, compress them (e.g., .zip or .tar.gz) and attach the archive under the "Anything else" section below.
📝 For a more detailed output that could help in troubleshooting, you may want to run Nuclei with the **`-verbose`** or **`-debug`** flags. This will provide additional insights into what's happening under the hood.
:warning: **Please redact any literal target hosts/URLs or other sensitive information.**
render: shell
@@ -64,9 +58,9 @@ body:
label: Environment
description: |
Examples:
- **OS**: Ubuntu 24.04
- **Nuclei** (`nuclei -version`): v3.6.0
- **Go** (`go version`): go1.24.0 _(only if you've installed it via the `go install` command)_
- **OS**: Ubuntu 20.04
- **Nuclei** (`nuclei -version`): v3.3.1
- **Go** (`go version`): go1.22.0 _(only if you've installed it via the `go install` command)_
value: |
- OS:
- Nuclei:
@@ -78,7 +72,7 @@ body:
attributes:
label: Anything else?
description: |
Links? References? Templates? CPU, memory, and trace profiles? Anything that will give us more context about the issue you are encountering!
Links? References? Templates? Anything that will give us more context about the issue you are encountering!
Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
validations:
+8 -12
View File
@@ -2,18 +2,14 @@ blank_issues_enabled: false
contact_links:
- name: 💡 Request a Feature (Start with Discussion)
url: https://github.com/orgs/projectdiscovery/discussions/new?category=ideas
about: Share your feature idea in discussions first. This helps validate and refine the request before creating an issue.
- name: Ask an question / advise on using nuclei
url: https://github.com/projectdiscovery/nuclei/discussions/categories/q-a
about: Ask a question or request support for using nuclei
- name: ❓ Ask Questions / Get Help
url: https://github.com/orgs/projectdiscovery/discussions
about: Get help and ask questions about using Nuclei. Many questions don't require issues.
- name: Share idea / feature to discuss for nuclei
url: https://github.com/projectdiscovery/nuclei/discussions/categories/ideas
about: Share idea / feature to discuss for nuclei
- name: 🔍 Browse Existing Issues
url: https://github.com/projectdiscovery/nuclei/issues
about: Check existing issues to see if your problem has already been reported or is being worked on.
- name: 💬 Connect with PD Team (Discord)
- name: Connect with PD Team (Discord)
url: https://discord.gg/projectdiscovery
about: Join our Discord for real-time discussions and community support on the #nuclei channel.
about: Connect with PD Team for direct communication
@@ -1,45 +0,0 @@
# Issue Template References
## Overview
This folder contains the preserved issue templates that are **not** directly accessible to users. These templates serve as references for maintainers when converting discussions to issues.
## New Workflow
### For Users:
1. **All reports start in Discussions** - Users cannot create issues directly
2. Bug reports go to [Q&A Discussions](https://github.com/projectdiscovery/nuclei/discussions/categories/q-a)
3. Feature requests go to [Ideas Discussions](https://github.com/projectdiscovery/nuclei/discussions/categories/ideas)
4. This helps filter out duplicate questions, invalid reports, and ensures proper triage
### For Maintainers:
1. **Review discussions** in both Q&A and Ideas categories
2. **Validate the reports** - ensure they're actual bugs/valid feature requests
3. **Use reference templates** when converting discussions to issues:
- Copy content from `bug-report-reference.yml` or `feature-request-reference.yml`
- Create a new issue manually with the appropriate template structure
- Link back to the original discussion
- Close the discussion or mark it as resolved
## Benefits
- **Better triage**: Avoid cluttering issues with questions and invalid reports
- **Community involvement**: Discussions allow for community input before creating issues
- **Quality control**: Maintainers can ensure issues follow proper format and contain necessary information
- **Reduced noise**: Only validated, actionable items become issues
## Reference Templates
- `bug-report-reference.yml` - Use when converting bug reports from discussions to issues
- `feature-request-reference.yml` - Use when converting feature requests from discussions to issues
## Converting a Discussion to Issue
1. Identify a valid discussion that needs to become an issue
2. Go to the main repository's Issues tab
3. Click "New Issue"
4. Manually create the issue using the reference template structure
5. Include all relevant information from the discussion
6. Add a comment linking back to the original discussion
7. Apply appropriate labels
8. Close or mark the discussion as resolved with a link to the created issue
-3
View File
@@ -2,9 +2,6 @@
<!-- Describe the overall picture of your modifications to help maintainers understand the pull request. PRs are required to be associated to their related issue tickets or feature request. -->
### Proof
<!-- How has this been tested? Please describe the tests that you ran to verify your changes. -->
## Checklist
-1
View File
@@ -2,7 +2,6 @@ addReviewers: true
reviewers:
- dogancanbakir
- dwisiswant0
- mzack9999
numberOfReviewers: 1
skipKeywords:
+17 -13
View File
@@ -1,24 +1,28 @@
name: 🔀 Auto merge PR
name: 🤖 Auto Merge
on:
# pull_request:
# types: [opened, synchronize, reopened, ready_for_review]
# pull_request_review:
# types: [submitted]
pull_request_review:
types: [submitted]
workflow_run:
workflows: ["♾️ Compatibility Checks"]
types: [completed]
workflows: ["♾️ Compatibility Check"]
types:
- completed
permissions:
contents: write
pull-requests: write
issues: write
repository-projects: write
jobs:
auto-merge-dependabot:
auto-merge:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
if: github.actor == 'dependabot[bot]'
steps:
- uses: projectdiscovery/actions/pr/approve@v1
- uses: projectdiscovery/actions/pr/merge@v1
- uses: actions/checkout@v4
with:
auto: "true"
token: ${{ secrets.DEPENDABOT_PAT }}
- uses: ahmadnassri/action-dependabot-auto-merge@v2
with:
github-token: ${{ secrets.DEPENDABOT_PAT }}
target: all
+3 -4
View File
@@ -7,14 +7,13 @@ on:
- dev
jobs:
compat-checks:
if: ${{ github.actor == 'dependabot[bot]' }}
check:
if: github.actor == 'dependabot[bot]'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go/compat-checks@v1
with:
go-version: "stable"
release-test: true
-61
View File
@@ -1,61 +0,0 @@
name: 📊 Flamegraph
on:
workflow_call: {}
jobs:
flamegraph:
name: "Flamegraph"
env:
PROFILE_MEM: "/tmp/nuclei-profile"
TARGET_URL: "http://honey.scanme.sh/-/?foo=bar"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- run: make build
- name: "Setup environment (push)"
if: ${{ github.event_name == 'push' }}
run: |
echo "PROFILE_MEM=${PROFILE_MEM}-${GITHUB_REF_NAME}-${GITHUB_SHA}" >> $GITHUB_ENV
echo "FLAMEGRAPH_NAME=nuclei-${GITHUB_REF_NAME} (${GITHUB_SHA})" >> $GITHUB_ENV
- name: "Setup environment (pull_request)"
if: ${{ github.event_name == 'pull_request' }}
run: |
echo "PROFILE_MEM=${PROFILE_MEM}-pr-${{ github.event.number }}" >> $GITHUB_ENV
echo "FLAMEGRAPH_NAME=nuclei (PR #${{ github.event.number }})" >> $GITHUB_ENV
- run: ./bin/nuclei -update-templates
- run: |
./bin/nuclei -silent -target="${TARGET_URL}" \
-disable-update-check \
-no-mhe -no-httpx \
-code -dast -file -headless \
-enable-self-contained -enable-global-matchers \
-rate-limit=0 \
-concurrency=250 \
-headless-concurrency=100 \
-payload-concurrency=250 \
-bulk-size=250 \
-headless-bulk-size=100 \
-profile-mem="${PROFILE_MEM}"
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-cpu
with:
profile: "${{ env.PROFILE_MEM }}.cpu"
name: "${{ env.FLAMEGRAPH_NAME }} CPU profiles"
continue-on-error: true
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-mem
with:
profile: "${{ env.PROFILE_MEM }}.mem"
name: "${{ env.FLAMEGRAPH_NAME }} memory profiles"
continue-on-error: true
- if: ${{ steps.flamegraph-mem.outputs.message == '' }}
run: |
echo "::notice::CPU flamegraph: ${{ steps.flamegraph-cpu.outputs.url }}"
echo "::notice::Memory (heap) flamegraph: ${{ steps.flamegraph-mem.outputs.url }}"
+3 -3
View File
@@ -4,14 +4,14 @@ on:
push:
branches:
- dev
workflow_dispatch: {}
workflow_dispatch:
jobs:
publish-docs:
if: ${{ !endsWith(github.actor, '[bot]') }}
if: "${{ !endsWith(github.actor, '[bot]') }}"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/setup/git@v1
- run: make syntax-docs
+38 -33
View File
@@ -1,50 +1,55 @@
name: 👤 Generate PGO
on:
workflow_dispatch: {}
workflow_call: {}
push:
branches: ["dev"]
paths:
- '**.go'
- '**.mod'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# TODO(dwisiswant0): https://go.dev/doc/pgo#merging-profiles
jobs:
pgo:
strategy:
matrix:
targets: [150]
runs-on: ubuntu-latest
if: github.repository == 'projectdiscovery/nuclei'
permissions:
contents: write
env:
PGO_FILE: "cmd/nuclei/default.pgo"
TARGET: "https://honey.scanme.sh"
TARGET_COUNT: "100"
TARGET_LIST: "/tmp/targets.txt"
PROFILE_MEM: "/tmp/nuclei-profile"
LIST_FILE: "/tmp/targets-${{ matrix.targets }}.txt"
PROFILE_MEM: "/tmp/nuclei-profile-${{ matrix.targets }}-targets"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/git@v1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- run: |
for i in {1..${{ env.TARGET_COUNT }}}; do
echo "${{ env.TARGET }}/-/?_=${i}" >> "${{ env.TARGET_LIST }}";
done
- run: make build
- run: ./bin/nuclei -update-templates
- run: |
./bin/nuclei -silent -list="${TARGET_LIST}" \
-disable-update-check \
-no-mhe -no-httpx \
-code -dast -file -headless \
-enable-self-contained -enable-global-matchers \
-rate-limit=0 \
-concurrency=250 \
-headless-concurrency=100 \
-payload-concurrency=250 \
-bulk-size=250 \
-headless-bulk-size=100 \
-profile-mem="${PROFILE_MEM}" \
env:
DISABLE_STDOUT: "1"
- run: mv "${PROFILE_MEM}.cpu" "${PGO_FILE}"
- uses: actions/upload-artifact@v6
- name: Generate list
run: for i in {1..${{ matrix.targets }}}; do echo "https://honey.scanme.sh/?_=${i}" >> "${LIST_FILE}"; done
# NOTE(dwisiswant0): use `-no-mhe` flag to get better samples.
- run: go run . -l "${LIST_FILE}" -profile-mem="${PROFILE_MEM}" -no-mhe
working-directory: cmd/nuclei/
- run: mv "${PROFILE_MEM}.cpu" ${PGO_FILE}
# NOTE(dwisiswant0): shall we prune $PGO_FILE git history?
# if we prune it, this won't be linear since it requires a force-push.
# if we don't, the git objects will just keep growing bigger.
#
# Ref:
# - https://go.dev/blog/pgo#:~:text=We%20recommend%20committing%20default.pgo%20files%20to%20your%20repository
# - https://gist.github.com/nottrobin/5758221
- uses: projectdiscovery/actions/commit@v1
with:
files: "${PGO_FILE}"
message: "build: update PGO profile :robot:"
- run: git push origin $GITHUB_REF
- uses: actions/upload-artifact@v4
with:
name: "pgo"
path: "${{ env.PGO_FILE }}"
overwrite: true
+3 -3
View File
@@ -3,7 +3,7 @@ name: 🐛 govulncheck
on:
schedule:
- cron: '0 0 * * 0' # Weekly
workflow_dispatch: {}
workflow_dispatch:
jobs:
govulncheck:
@@ -16,11 +16,11 @@ jobs:
env:
OUTPUT: "/tmp/results.sarif"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- run: go install golang.org/x/vuln/cmd/govulncheck@latest
- run: govulncheck -scan package -format sarif ./... > $OUTPUT
- uses: github/codeql-action/upload-sarif@v4
- uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: "${{ env.OUTPUT }}"
category: "govulncheck"
+5 -7
View File
@@ -1,8 +1,8 @@
name: 🔨 Performance Regression
on:
workflow_call: {}
workflow_dispatch: {}
workflow_call:
workflow_dispatch:
jobs:
perf-regression:
@@ -11,15 +11,13 @@ jobs:
env:
BENCH_OUT: "/tmp/bench.out"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- run: make build-test
- run: ./bin/nuclei.test -test.run - -test.bench=. -test.benchmem ./cmd/nuclei/ | tee $BENCH_OUT
env:
DISABLE_STDOUT: "1"
- uses: actions/cache/restore@v5
- uses: actions/cache/restore@v4
with:
path: ./cache
key: ${{ runner.os }}-benchmark
@@ -33,7 +31,7 @@ jobs:
github-token: ${{ secrets.GITHUB_TOKEN }}
comment-on-alert: true
summary-always: true
- uses: actions/cache/save@v5
- uses: actions/cache/save@v4
if: github.event_name == 'push'
with:
path: ./cache
+43
View File
@@ -0,0 +1,43 @@
name: 🔨 Performance Test
on:
schedule:
- cron: '0 0 * * 0' # Weekly
workflow_dispatch:
jobs:
perf-test:
strategy:
matrix:
count: [50, 100, 150]
runs-on: ubuntu-latest
if: github.repository == 'projectdiscovery/nuclei'
env:
LIST_FILE: "/tmp/targets-${{ matrix.count }}.txt"
PROFILE_MEM: "/tmp/nuclei-perf-test-${{ matrix.count }}"
steps:
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- run: make verify
- name: Generate list
run: for i in {1..${{ matrix.count }}}; do echo "https://honey.scanme.sh/?_=${i}" >> "${LIST_FILE}"; done
- run: go run . -l "${LIST_FILE}" -profile-mem="${PROFILE_MEM}"
env:
NUCLEI_ARGS: host-error-stats
working-directory: cmd/nuclei/
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-cpu
with:
profile: "${{ env.PROFILE_MEM }}.cpu"
name: "${{ env.FLAMEGRAPH_NAME }} CPU profiles"
continue-on-error: true
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-mem
with:
profile: "${{ env.PROFILE_MEM }}.mem"
name: "${{ env.FLAMEGRAPH_NAME }} memory profiles"
continue-on-error: true
- if: ${{ steps.flamegraph-mem.outputs.message == '' }}
run: |
echo "::notice::CPU flamegraph: ${{ steps.flamegraph-cpu.outputs.url }}"
echo "::notice::Memory (heap) flamegraph: ${{ steps.flamegraph-mem.outputs.url }}"
+4 -16
View File
@@ -4,28 +4,16 @@ on:
push:
tags:
- '*'
workflow_dispatch: {}
workflow_dispatch:
jobs:
pgo:
name: "Generate PGO"
uses: ./.github/workflows/generate-pgo.yaml
release:
name: "Release"
needs: ["pgo"]
jobs:
release:
runs-on: ubuntu-latest-16-cores
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/download-artifact@v7
with:
name: "pgo"
path: "cmd/nuclei/"
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version: "stable"
- uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
+3 -2
View File
@@ -13,7 +13,7 @@ jobs:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v10
- uses: actions/stale@v9
with:
days-before-stale: 90
days-before-close: 7
@@ -37,4 +37,5 @@ jobs:
it, please comment or feel free to reopen it.
close-issue-label: "Status: Abandoned"
close-pr-label: "Status: Abandoned"
exempt-issue-labels: "Type: Enhancement"
exempt-issue-labels: "Status: Abandoned"
exempt-pr-labels: "Status: Abandoned"
+51 -41
View File
@@ -19,13 +19,12 @@ concurrency:
jobs:
lint:
name: "Lint"
if: ${{ !endsWith(github.actor, '[bot]') }}
if: "${{ !endsWith(github.actor, '[bot]') }}"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/golangci-lint/v2@v1
- uses: projectdiscovery/actions/golangci-lint@v1
tests:
name: "Tests"
@@ -36,19 +35,8 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: "${{ matrix.os }}"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- uses: projectdiscovery/actions/free-disk-space@v1
with:
llvm: 'false'
php: 'false'
mongodb: 'false'
mysql: 'false'
misc-packages: 'false'
docker-images: 'false'
tools-cache: 'false'
- run: make vet
- run: make build
- run: make test
@@ -64,20 +52,16 @@ jobs:
needs: ["tests"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- name: "Simple"
run: go run .
working-directory: examples/simple/
# - run: go run . # Temporarily disabled very flaky in github actions
# working-directory: examples/advanced/
# TODO: FIX with ExecutionID (ref: https://github.com/projectdiscovery/nuclei/pull/6296)
# - name: "with Speed Control"
# run: go run .
# working-directory: examples/with_speed_control/
- name: "with Speed Control"
run: go run .
working-directory: examples/with_speed_control/
integration:
name: "Integration tests"
@@ -88,11 +72,9 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- uses: projectdiscovery/actions/setup/python@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- run: bash run.sh "${{ matrix.os }}"
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
@@ -109,11 +91,9 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/nuclei@v1
- uses: projectdiscovery/actions/setup/python@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- run: bash run.sh
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
@@ -124,9 +104,8 @@ jobs:
needs: ["tests"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/cache/go-rod-browser@v1
- run: make template-validate
codeql:
@@ -138,30 +117,61 @@ jobs:
contents: read
security-events: write
steps:
- uses: actions/checkout@v6
- uses: github/codeql-action/init@v4
- uses: actions/checkout@v4
- uses: github/codeql-action/init@v3
with:
languages: 'go'
- uses: github/codeql-action/autobuild@v4
- uses: github/codeql-action/analyze@v4
- uses: github/codeql-action/autobuild@v3
- uses: github/codeql-action/analyze@v3
release:
name: "Release test"
needs: ["tests"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version: "stable"
- uses: projectdiscovery/actions/goreleaser@v1
flamegraph:
name: "Flamegraph"
needs: ["tests"]
uses: ./.github/workflows/flamegraph.yaml
env:
PROFILE_MEM: "/tmp/nuclei"
TARGET_URL: "http://scanme.sh/a/?b=c"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: make build
- name: "Setup environment (push)"
if: ${{ github.event_name == 'push' }}
run: |
echo "PROFILE_MEM=${PROFILE_MEM}-${GITHUB_REF_NAME}-${GITHUB_SHA}" >> $GITHUB_ENV
echo "FLAMEGRAPH_NAME=nuclei-${GITHUB_REF_NAME} (${GITHUB_SHA})" >> $GITHUB_ENV
- name: "Setup environment (pull_request)"
if: ${{ github.event_name == 'pull_request' }}
run: |
echo "PROFILE_MEM=${PROFILE_MEM}-pr-${{ github.event.number }}" >> $GITHUB_ENV
echo "FLAMEGRAPH_NAME=nuclei (PR #${{ github.event.number }})" >> $GITHUB_ENV
- run: ./bin/nuclei -silent -update-templates
- run: ./bin/nuclei -silent -u "${TARGET_URL}" -profile-mem="${PROFILE_MEM}"
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-cpu
with:
profile: "${{ env.PROFILE_MEM }}.cpu"
name: "${{ env.FLAMEGRAPH_NAME }} CPU profiles"
continue-on-error: true
- uses: projectdiscovery/actions/flamegraph@v1
id: flamegraph-mem
with:
profile: "${{ env.PROFILE_MEM }}.mem"
name: "${{ env.FLAMEGRAPH_NAME }} memory profiles"
continue-on-error: true
- if: ${{ steps.flamegraph-mem.outputs.message == '' }}
run: |
echo "::notice::CPU flamegraph: ${{ steps.flamegraph-cpu.outputs.url }}"
echo "::notice::Memory (heap) flamegraph: ${{ steps.flamegraph-mem.outputs.url }}"
perf-regression:
name: "Performance regression"
needs: ["tests"]
uses: ./.github/workflows/perf-regression.yaml
+1 -3
View File
@@ -8,7 +8,7 @@
.gitignore
.idea
.vscode
.claude/settings.local.json
# Binaries
/bin/*
/bindgen
@@ -28,8 +28,6 @@
/scrapefunc
/scrapefuncs
/tsgen
/integration_tests/integration-test
/integration_tests/nuclei
# Templates
/*.yaml
+4 -5
View File
@@ -11,9 +11,8 @@ builds:
id: nuclei-cli
env:
- CGO_ENABLED=0
- GOEXPERIMENT=greenteagc
goos: [windows, linux, darwin]
goarch: [amd64, '386', arm, arm64]
goos: [windows,linux,darwin]
goarch: [amd64,'386',arm,arm64]
ignore:
- goos: darwin
goarch: '386'
@@ -39,9 +38,9 @@ builds:
# goarch: [amd64]
archives:
- formats: [zip]
- format: zip
id: nuclei
ids: [nuclei-cli]
builds: [nuclei-cli]
name_template: '{{ .ProjectName }}_{{ .Version }}_{{ if eq .Os "darwin" }}macOS{{ else }}{{ .Os }}{{ end }}_{{ .Arch }}'
checksum:
-83
View File
@@ -1,83 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Nuclei is a modern, high-performance vulnerability scanner built in Go that leverages YAML-based templates for customizable vulnerability detection. It supports multiple protocols (HTTP, DNS, TCP, SSL, WebSocket, WHOIS, JavaScript, Code) and is designed for zero false positives through real-world condition simulation.
## Development Commands
### Building and Testing
- `make build` - Build the main nuclei binary to ./bin/nuclei
- `make test` - Run unit tests with race detection
- `make integration` - Run integration tests (builds and runs test suite)
- `make functional` - Run functional tests
- `make vet` - Run go vet for code analysis
- `make tidy` - Clean up go modules
### Validation and Linting
- `make template-validate` - Validate nuclei templates using the built binary
- `go fmt ./...` - Format Go code
- `go vet ./...` - Static analysis
### Development Tools
- `make devtools-all` - Build all development tools (bindgen, tsgen, scrapefuncs)
- `make jsupdate-all` - Update JavaScript bindings and TypeScript definitions
- `make docs` - Generate documentation
- `make memogen` - Generate memoization code for JavaScript libraries
### Testing Specific Components
- Run single test: `go test -v ./pkg/path/to/package -run TestName`
- Integration tests are in `integration_tests/` and can be run via `make integration`
## Architecture Overview
### Core Components
- **cmd/nuclei** - Main CLI entry point with flag parsing and configuration
- **internal/runner** - Core runner that orchestrates the entire scanning process
- **pkg/core** - Execution engine with work pools and template clustering
- **pkg/templates** - Template parsing, compilation, and management
- **pkg/protocols** - Protocol implementations (HTTP, DNS, Network, etc.)
- **pkg/operators** - Matching and extraction logic (matchers/extractors)
- **pkg/catalog** - Template discovery and loading from disk/remote sources
### Protocol Architecture
Each protocol (HTTP, DNS, Network, etc.) implements:
- Request interface with Compile(), ExecuteWithResults(), Match(), Extract() methods
- Operators embedding for matching/extraction functionality
- Protocol-specific request building and execution logic
### Template System
- Templates are YAML files defining vulnerability detection logic
- Compiled into executable requests with operators (matchers/extractors)
- Support for workflows (multistep template execution)
- Template clustering optimizes identical requests across multiple templates
### Key Execution Flow
1. Template loading and compilation via pkg/catalog/loader
2. Input provider setup for targets
3. Engine creation with work pools for concurrency
4. Template execution with result collection via operators
5. Output writing and reporting integration
### JavaScript Integration
- Custom JavaScript runtime for code protocol templates
- Auto-generated bindings in pkg/js/generated/
- Library implementations in pkg/js/libs/
- Development tools for binding generation in pkg/js/devtools/
## Template Development
- Templates located in separate nuclei-templates repository
- YAML format with info, requests, and operators sections
- Support for multiple protocol types in single template
- Built-in DSL functions for dynamic content generation
- Template validation available via `make template-validate`
## Key Directories
- **lib/** - SDK for embedding nuclei as a library
- **examples/** - Usage examples for different scenarios
- **integration_tests/** - Integration test suite with protocol-specific tests
- **pkg/fuzz/** - Fuzzing engine and DAST capabilities
- **pkg/input/** - Input processing for various formats (Burp, OpenAPI, etc.)
- **pkg/reporting/** - Result export and issue tracking integrations
+1 -3
View File
@@ -22,7 +22,7 @@ When this flag is provided, nuclei will print the list of all helper functions a
- `-svd` flag
When this flag is provided, nuclei will print all `variables` pre- and post-execution of a request for a template. This is useful to understand what variables are available for a template and what values they have.
When this flag is provided, nuclei will print all `variables` pre and post execution of a request for a template. This is useful to understand what variables are available for a template and what values they have.
- `-elog = errors.txt` flag
@@ -40,8 +40,6 @@ Nuclei was built with some environment variables in mind to help with debugging.
| `SHOW_DSL_ERRORS=true` | Enables Printing DSL Errors (that are hidden by default) |
| `HIDE_TEMPLATE_SIG_WARNING=true` | Hides Template Signature Verification Warnings |
| `NUCLEI_LOG_ALL=true` | Log All Events that were skipped in verbose mode |
| `NUCLEI_CONFIG_DIR` | Sets custom configuration directory path |
| `NUCLEI_TEMPLATES_DIR` | Sets custom nuclei templates directory path |
+1 -1
View File
@@ -145,7 +145,7 @@ type Exporter interface {
}
```
Exporters include `Elasticsearch`, `markdown`, `sarif` . Trackers include `GitHub`, `GitLab` and `Jira`.
Exporters include `Elasticsearch`, `markdown`, `sarif` . Trackers include `GitHub` , `GitLab` and `Jira`.
Each exporter and trackers implement their own configuration in YAML format and are very modular in nature, so adding new ones is easy.
+1 -1
View File
@@ -1,5 +1,5 @@
# Build
FROM golang:1.24-alpine AS builder
FROM golang:1.23-alpine AS builder
RUN apk add build-base
WORKDIR /app
+5 -14
View File
@@ -15,8 +15,8 @@ ifneq ($(shell go env GOOS),darwin)
endif
.PHONY: all build build-stats clean devtools-all devtools-bindgen devtools-scrapefuncs
.PHONY: devtools-tsgen docs docgen dsl-docs functional fuzzplayground go-build lint lint-strict syntax-docs
.PHONY: integration jsupdate-all jsupdate-bindgen jsupdate-tsgen memogen scan-charts test test-with-lint
.PHONY: devtools-tsgen docs docgen dsl-docs functional fuzzplayground go-build syntax-docs
.PHONY: integration jsupdate-all jsupdate-bindgen jsupdate-tsgen memogen scan-charts test
.PHONY: tidy ts verify download vet template-validate
all: build
@@ -75,7 +75,7 @@ syntax-docs: docgen
syntax-docs:
./bin/docgen SYNTAX-REFERENCE.md nuclei-jsonschema.json
test: GOFLAGS = -race -v -timeout 30m -count 1
test: GOFLAGS = -race -v
test:
$(GOTEST) $(GOFLAGS) ./...
@@ -146,14 +146,5 @@ dsl-docs:
template-validate: build
template-validate:
./bin/nuclei -ut
./bin/nuclei -validate \
-et http/technologies \
-t dns \
-t ssl \
-t network \
-t http/exposures \
-ept code
./bin/nuclei -validate \
-w workflows \
-et http/technologies \
-ept code
./bin/nuclei -validate -et http/technologies
./bin/nuclei -validate -w workflows -et http/technologies
+8 -8
View File
@@ -14,6 +14,8 @@
<p align="center">
<img src="https://img.shields.io/badge/go-1.22-%2300ADD8.svg?style=for-the-badge&logo=go&logoColor=white">
&nbsp;&nbsp;
<a href="https://docs.projectdiscovery.io/tools/nuclei/overview?utm_source=github&utm_medium=web&utm_campaign=nuclei_readme"><img src="https://img.shields.io/badge/Documentation-%23000000.svg?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9IiNmZmZmZmYiIHN0cm9rZS13aWR0aD0iMiIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIiBzdHJva2UtbGluZWpvaW49InJvdW5kIiBjbGFzcz0ibHVjaWRlIGx1Y2lkZS1ib29rLW9wZW4iPjxwYXRoIGQ9Ik0xMiA3djE0Ii8+PHBhdGggZD0iTTMgMThhMSAxIDAgMCAxLTEtMVY0YTEgMSAwIDAgMSAxLTFoNWE0IDQgMCAwIDEgNCA0IDQgNCAwIDAgMSA0LTRoNWExIDEgMCAwIDEgMSAxdjEzYTEgMSAwIDAgMS0xIDFoLTZhMyAzIDAgMCAwLTMgMyAzIDMgMCAwIDAtMy0zeiIvPjwvc3ZnPg==&logoColor=white"></a>
&nbsp;&nbsp;
<a href="https://github.com/projectdiscovery/nuclei-templates"><img src="https://img.shields.io/badge/Templates Library-%23000000.svg?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxNiIgaGVpZ2h0PSIxNiIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9IiNmZmZmZmYiIHN0cm9rZS13aWR0aD0iMS41IiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJsdWNpZGUgbHVjaWRlLXNoaWVsZCI+PHBhdGggZD0iTTIwIDEzYzAgNS0zLjUgNy41LTcuNjYgOC45NWExIDEgMCAwIDEtLjY3LS4wMUM3LjUgMjAuNSA0IDE4IDQgMTNWNmExIDEgMCAwIDEgMS0xYzIgMCA0LjUtMS4yIDYuMjQtMi43MmExLjE3IDEuMTcgMCAwIDEgMS41MiAwQzE0LjUxIDMuODEgMTcgNSAxOSA1YTEgMSAwIDAgMSAxIDF6Ii8+PC9zdmc+&logoColor=white"></a>
@@ -33,7 +35,7 @@
- Reduce false positives by simulating real-world steps to verify a vulnerability.
- Ultra-fast parallel scan processing and request clustering.
- Integrate into CI/CD pipelines for vulnerability detection and regression testing.
- Supports multiple protocols like TCP, DNS, HTTP, SSL, WHOIS, JavaScript, Code and more.
- Supports multiple protocols like TCP, DNS, HTTP, SSL, WHOIS JavaScript, Code and more.
- Integrate with Jira, Splunk, GitHub, Elastic, GitLab.
<br>
@@ -83,7 +85,7 @@ _For security teams and enterprises, we provide a cloud-hosted service built on
- 50x faster scans
- Large scale scanning with high accuracy
- Integrations with cloud services (AWS, GCP, Azure, Cloudflare, Fastly, Terraform, Kubernetes)
- Integrations with cloud services (AWS, GCP, Azure, CloudFlare, Fastly, Terraform, Kubernetes)
- Jira, Slack, Linear, APIs and Webhooks
- Executive and compliance reporting
- Plus: Real-time scanning, SAML SSO, SOC 2 compliant platform (with EU and US hosting options), shared team workspaces, and more
@@ -97,7 +99,7 @@ _For security teams and enterprises, we provide a cloud-hosted service built on
## Documentation
Browse the full Nuclei [**`documentation here`**](https://docs.projectdiscovery.io/tools/nuclei/running). If you’re new to Nuclei, check out our [**`foundational YouTube series`**](https://www.youtube.com/playlist?list=PLZRbR9aMzTTpItEdeNSulo8bYsvil80Rl).
Browse the full Nuclei [**`documentation here`**](https://docs.projectdiscovery.io/tools/nuclei/running). If you’re new to Nuclei, check out our [**`foundational Youtube series`**](https://www.youtube.com/playlist?list=PLZRbR9aMzTTpItEdeNSulo8bYsvil80Rl).
<div align="center">
@@ -109,7 +111,7 @@ Browse the full Nuclei [**`documentation here`**](https://docs.projectdiscovery.
### Installation
`nuclei` requires **go >= 1.24.2** to install successfully. Run the following command to get the repo:
`nuclei` requires **go1.23** to install successfully. Run the following command to get the repo:
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -140,7 +142,7 @@ TARGET:
-u, -target string[] target URLs/hosts to scan
-l, -list string path to file containing a list of target URLs/hosts to scan (one per line)
-eh, -exclude-hosts string[] hosts to exclude to scan from the input list (ip, cidr, hostname)
-resume string resume scan from and save to specified file (clustering will be disabled)
-resume string resume scan using resume.cfg (clustering will be disabled)
-sa, -scan-all-ips scan all the IP's associated with dns record
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
@@ -161,7 +163,7 @@ TEMPLATES:
-validate validate the passed templates to nuclei
-nss, -no-strict-syntax disable strict syntax check on templates
-td, -template-display displays the templates content
-tl list all templates matching current filters
-tl list all available templates
-tgl list all available tags
-sign signs the templates with the private key defined in NUCLEI_SIGNATURE_PRIVATE_KEY env variable
-code enable loading code protocol-based templates
@@ -284,7 +286,6 @@ RATE-LIMIT:
-jsc, -js-concurrency int maximum number of javascript runtimes to be executed in parallel (default 120)
-pc, -payload-concurrency int max payload concurrency for each template (default 25)
-prc, -probe-concurrency int http probe concurrency with httpx (default 50)
-tlc, -template-loading-concurrency int maximum number of concurrent template loading operations (default 50)
OPTIMIZATIONS:
-timeout int time to wait in seconds before timeout (default 10)
@@ -355,7 +356,6 @@ CLOUD:
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
EXAMPLES:
+3 -4
View File
@@ -59,7 +59,7 @@ Nuclei使用零误报的定制模板向目标发送请求,同时可以对主
# 安装Nuclei
Nuclei需要 **go1.24.2** 才能安装成功。执行下列命令安装最新版本的Nuclei
Nuclei需要 **go1.22** 才能安装成功。执行下列命令安装最新版本的Nuclei
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -119,7 +119,7 @@ Nuclei是一款注重于可配置性、可扩展性和易用性的基于模板
目标:
-u, -target string[] 指定扫描的目标URL/主机(多个目标则指定多个-u参数)
-l, -list string 指定包含要扫描的目标URL/主机列表的文件路径(一行一个)
-resume string 从指定文件恢复扫描并保存到指定文件(将禁用请求聚类)
-resume string 使用指定的resume.cfg文件恢复扫描(将禁用请求聚类)
-sa, -scan-all-ips 扫描由目标解析出来的所有IP(针对域名对应多个IP的情况)
-iv, -ip-version string[] 要扫描的主机名的IP版本(4,6)-(默认为4)
@@ -238,7 +238,6 @@ UNCOVER引擎:
-c, -concurrency int 并行执行的最大模板数量(默认:25)
-hbs, -headless-bulk-size int 每个模板并行运行的无头主机最大数量(默认:10)
-headc, -headless-concurrency int 并行指定无头主机最大数量(默认:10)
-tlc, -template-loading-concurrency int 最大并发模板加载操作数(默认:50)
优化:
@@ -371,7 +370,7 @@ Nuclei允许您定制自己的测试方法,可以轻松的运行您的程序
- 可以在几分钟处理上千台主机
- 使用YAML语法定制自动化测试
欢迎查看我们其他的开源项目,可能有适合您的赏金猎人工作流:[github.com/projectdiscovery](https://github.com/projectdiscovery),我们还使用[Chaos绘制了每日的DNS数据](https://chaos.projectdiscovery.io)。
欢迎查看我们其他的开源项目,可能有适合您的赏金猎人工作流:[github.com/projectdiscovery](http://github.com/projectdiscovery),我们还使用[Chaos绘制了每日的DNS数据](http://chaos.projectdiscovery.io)。
</td>
</tr>
+2 -3
View File
@@ -56,7 +56,7 @@ Tenemos un [repositorio dedicado](https://github.com/projectdiscovery/nuclei-tem
# Instalación de Nuclei
Nuclei requiere **go1.24.2** para instalarse correctamente. Ejecute el siguiente comando para instalar la última versión -
Nuclei requiere **go1.22** para instalarse correctamente. Ejecute el siguiente comando para instalar la última versión -
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -118,7 +118,7 @@ TARGET:
-u, -target string[] URLs/hosts a escanear
-l, -list string ruta al archivo que contiene la lista de URLs/hosts a escanear (uno por línea)
-eh, -exclude-hosts string[] hosts a excluir para escanear de la lista de entrada (ip, cidr, hostname)
-resume string reanudar el escaneo desde y guardar en el archivo especificado (la clusterización quedará inhabilitada)
-resume string reanudar el escaneo usando resume.cfg (la clusterización quedará inhabilitada)
-sa, -scan-all-ips escanear todas las IP asociadas al registro dns
-iv, -ip-version string[] versión IP a escanear del nombre de host (4,6) - (por defecto 4)
@@ -244,7 +244,6 @@ RATE-LIMIT:
-headc, -headless-concurrency int número máximo de plantillas headless a ejecutar en paralelo (por defecto 10)
-jsc, -js-concurrency int número máximo de entornos de ejecución de JavaScript a ejecutar en paralelo (por defecto 120)
-pc, -payload-concurrency int concurrencia máxima de carga útil para cada plantilla (por defecto 25)
-tlc, -template-loading-concurrency int número máximo de operaciones de carga de plantillas concurrentes (por defecto 50)
OPTIMIZATIONS:
-timeout int tiempo de espera en segundos (por defecto 10)
+3 -4
View File
@@ -54,7 +54,7 @@ Kami memiliki [repositori khusus](https://github.com/projectdiscovery/nuclei-tem
# Instalasi Nuclei
Nuclei membutuhkan **go1.24.2** agar dapat diinstall. Jalankan perintah berikut untuk menginstal versi terbaru -
Nuclei membutuhkan **go1.22** agar dapat diinstall. Jalankan perintah berikut untuk menginstal versi terbaru -
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -98,7 +98,7 @@ Flags:
TARGET:
-u, -target string[] target URLs/hosts to scan
-l, -list string path to file containing a list of target URLs/hosts to scan (one per line)
-resume string resume scan from and save to specified file (clustering will be disabled)
-resume string resume scan using resume.cfg (clustering will be disabled)
-sa, -scan-all-ips scan all the IP's associated with dns record
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
@@ -211,7 +211,6 @@ RATE-LIMIT:
-c, -concurrency int maximum number of templates to be executed in parallel (default 25)
-hbs, -headless-bulk-size int maximum number of headless hosts to be analyzed in parallel per template (default 10)
-headc, -headless-concurrency int maximum number of headless templates to be executed in parallel (default 10)
-tlc, -template-loading-concurrency int maximum number of concurrent template loading operations (default 50)
OPTIMIZATIONS:
-timeout int time to wait in seconds before timeout (default 10)
@@ -342,7 +341,7 @@ Nuclei memungkinkan Anda untuk menyesuaikan pendekatan pengujian Anda dengan ran
- Dapat memproses ribuan host hanya dalam beberapa menit.
- Mudah mengotomatiskan pendekatan pengujian khusus Anda dengan sintaks DSL berbasis YAML sederhana kami.
Silakan periksa proyek sumber terbuka kami yang lain yang mungkin cocok dengan alur kerja celah berhadiah Anda: [github.com/projectdiscovery](https://github.com/projectdiscovery), kami juga menyediakan [penyegaran data DNS di Chaos setiap hari](https://chaos.projectdiscovery.io).
Silakan periksa proyek sumber terbuka kami yang lain yang mungkin cocok dengan alur kerja celah berhadiah Anda: [github.com/projectdiscovery](http://github.com/projectdiscovery), kami juga menyediakan [penyegaran data DNS di Chaos setiap hari](http://chaos.projectdiscovery.io).
</td>
</tr>
+2 -2
View File
@@ -53,7 +53,7 @@ Nucleiは、テンプレートに基づいてターゲット間でリクエス
# Nucleiのインストール
Nucleiを正常にインストールするには、**go1.24.2**が必要です。以下のコマンドを実行して最新バージョンをインストールしてください -
Nucleiを正常にインストールするには、**go1.22**が必要です。以下のコマンドを実行して最新バージョンをインストールしてください -
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -113,7 +113,7 @@ Nucleiは、広範な設定可能性、大規模な拡張性、および使い
ターゲット:
-u, -target string[] スキャンする対象のURL/ホスト
-l, -list string スキャンする対象のURL/ホストのリストが含まれているファイルへのパス(1行に1つ)
-resume string 指定されたファイルからスキャンを再開し、指定されたファイルに保存(クラスタリングは無効になります)
-resume string resume.cfgを使用してスキャンを再開(クラスタリングは無効になります)
-sa, -scan-all-ips DNSレコードに関連付けられているすべてのIPをスキャン
-iv, -ip-version string[] ホスト名のスキャンするIPバージョン(4,6)-(デフォルトは4)
+3 -4
View File
@@ -52,7 +52,7 @@ Nuclei는 템플릿을 기반으로 대상 간에 요청을 보내기 위해 사
# 설치
Nuclei를 성공적으로 설치하기 위해서 **go1.24.2**가 필요합니다. 다음 명령을 실행하여 최신 버전을 설치합니다.
Nuclei를 성공적으로 설치하기 위해서 **go1.22**가 필요합니다. 다음 명령을 실행하여 최신 버전을 설치합니다.
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -96,7 +96,7 @@ Nuclei는 빠르고, 템플릿 기반의 취약점 스캐너로
TARGET:
-u, -target string[] 스캔할 대상 URL/호스트
-l, -list string 스캔할 대상 URL/호스트 목록이 있는 파일 경로 (한 줄에 하나씩)
-resume string 지정된 파일에서 스캔을 재개하고 지정된 파일에 저장 (클러스터링은 비활성화됨)
-resume string resume.cfg를 사용하여 스캔 재개 (클러스터링은 비활성화됨)
-sa, -scan-all-ips dns 레코드와 관련된 모든 IP 스캔
-iv, -ip-version string[] 스캔할 호스트의 IP 버전 (4,6) - (기본값 4)
@@ -209,7 +209,6 @@ RATE-LIMIT:
-c, -concurrency int 병렬로 실행할 최대 템플릿 수 (기본값 25)
-hbs, -headless-bulk-size int 템플릿당 병렬로 분석할 최대 headless 호스트 수 (기본값 10)
-headc, -headless-concurrency int 병렬로 실행할 최대 headless 템플릿 수 (기본값 10)
-tlc, -template-loading-concurrency int 최대 동시 템플릿 로딩 작업 수 (기본값 50)
OPTIMIZATIONS:
-timeout int 타임아웃 전에 기다릴 초 수 (기본값 10)
@@ -342,7 +341,7 @@ Nuclei를 사용하면 자체 검사 모음으로 테스트 접근 방식을 사
- 몇 분 안에 수천 개의 호스트를 처리할 수 있음.
- 간단한 YAML DSL로 사용자 지정 테스트 접근 방식을 쉽게 자동화할 수 있음.
버그 바운티 워크플로에 맞는 다른 오픈 소스 프로젝트를 확인할 수 있습니다.: [github.com/projectdiscovery](https://github.com/projectdiscovery), 또한, 우리는 매일 [Chaos에서 DNS 데이터를 갱신해 호스팅합니다](https://chaos.projectdiscovery.io).
버그 바운티 워크플로에 맞는 다른 오픈 소스 프로젝트를 확인할 수 있습니다.: [github.com/projectdiscovery](http://github.com/projectdiscovery), 또한, 우리는 매일 [Chaos에서 DNS 데이터를 갱신해 호스팅합니다.](http://chaos.projectdiscovery.io).
</td>
</tr>
+2 -3
View File
@@ -56,7 +56,7 @@ Temos um [repositório dedicado](https://github.com/projectdiscovery/nuclei-temp
# Instalação do Nuclei
O Nuclei requer **go1.24.2** para ser instalado corretamente. Execute o seguinte comando para instalar a versão mais recente:
O Nuclei requer **go1.22** para ser instalado corretamente. Execute o seguinte comando para instalar a versão mais recente:
```sh
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
@@ -118,7 +118,7 @@ TARGET:
-u, -target string[] URLs/hosts a serem escaneados
-l, -list string caminho do arquivo contendo a lista de URLs/hosts a serem escaneados (um por linha)
-eh, -exclude-hosts string[] hosts a serem excluídos do escaneamento na lista de entrada (ip, cidr, hostname)
-resume string retomar o escaneamento a partir de e salvar no arquivo especificado (a clusterização será desabilitada)
-resume string retomar o escaneamento usando resume.cfg (a clusterização será desabilitada)
-sa, -scan-all-ips escanear todos os IPs associados ao registro DNS
-iv, -ip-version string[] versão de IP a escanear do nome do host (4,6) - (padrão 4)
@@ -244,7 +244,6 @@ RATE-LIMIT:
-headc, -headless-concurrency int número máximo de templates headless a serem executados em paralelo (padrão 10)
-jsc, -js-concurrency int número máximo de ambientes de execução de JavaScript a serem executados em paralelo (padrão 120)
-pc, -payload-concurrency int concorrência máxima de payload para cada template (padrão 25)
-tlc, -template-loading-concurrency int número máximo de operações de carregamento de templates concorrentes (padrão 50)
OPTIMIZATIONS:
-timeout int tempo limite em segundos (padrão 10)
+2 -2
View File
@@ -3218,7 +3218,7 @@ Inputs contains inputs for the network socket
<div class="dt">
description: |
Port is the port to send network requests to. this acts as default port but is overridden if target/input contains
Port is the port to send network requests to. this acts as default port but is overriden if target/input contains
non-http(s) ports like 80,8080,8081 etc
</div>
@@ -4512,7 +4512,7 @@ permutations and combinations for all payloads.
</div>
<div class="dt">
Payload concurrency i.e threads for sending requests.
Payload concurreny i.e threads for sending requests.
+1 -3
View File
@@ -19,9 +19,7 @@ func writeToFile(filename string, data []byte) {
if err != nil {
log.Fatalf("Could not create file %s: %s\n", filename, err)
}
defer func() {
_ = file.Close()
}()
defer file.Close()
_, err = file.Write(data)
if err != nil {
+1 -3
View File
@@ -41,9 +41,7 @@ func runFunctionalTests(debug bool) (error, bool) {
if err != nil {
return errors.Wrap(err, "could not open test cases"), true
}
defer func() {
_ = file.Close()
}()
defer file.Close()
errored, failedTestCases := runTestCases(file, debug)
+15 -15
View File
@@ -36,24 +36,24 @@
{{binary}} -tags cve,dos,fuzz
{{binary}} -tags cve -include-tags dos,fuzz
{{binary}} -tags cve -exclude-tags cve2020
{{binary}} -tags cve -exclude-templates http/cves/2020/
{{binary}} -tags cve -exclude-templates http/cves/2020/CVE-2020-9757.yaml
{{binary}} -tags cve -exclude-templates http/cves/2020/CVE-2020-9757.yaml -exclude-templates http/cves/2021/
{{binary}} -t http/cves/
{{binary}} -t http/cves/ -t http/exposures/
{{binary}} -t http/cves/ -t http/exposures/ -tags config
{{binary}} -t http/cves/ -t http/exposures/ -tags config,ssrf
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical -author geeknik,pdteam
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -exclude-templates http/cves/2021/
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -exclude-templates http/cves/2017/CVE-2017-7269.yaml
{{binary}} -t http/cves/ -t http/exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -include-templates http/cves/2017/CVE-2017-7269.yaml
{{binary}} -tags cve -exclude-templates cves/2020/
{{binary}} -tags cve -exclude-templates cves/2020/CVE-2020-9757.yaml
{{binary}} -tags cve -exclude-templates cves/2020/CVE-2020-9757.yaml -exclude-templates cves/2021/
{{binary}} -t cves/
{{binary}} -t cves/ -t exposures/
{{binary}} -t cves/ -t exposures/ -tags config
{{binary}} -t cves/ -t exposures/ -tags config,ssrf
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical -author geeknik,pdteam
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -exclude-templates cves/2021/
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -exclude-templates cves/2017/CVE-2017-7269.yaml
{{binary}} -t cves/ -t exposures/ -tags config -severity high,critical -author geeknik,pdteam -etags sqli -include-templates cves/2017/CVE-2017-7269.yaml
# Advanced Filtering
{{binary}} -tags cve -author geeknik,pdteam -tc severity=='high'
{{binary}} -tc contains(authors,'pdteam')
{{binary}} -t http/cves/ -t http/exposures/ -tc contains(tags,'cve') -exclude-templates http/cves/2020/CVE-2020-9757.yaml
{{binary}} -t cves/ -t exposures/ -tc contains(tags,'cve') -exclude-templates cves/2020/CVE-2020-9757.yaml
{{binary}} -tc protocol=='dns'
{{binary}} -tc contains(http_method,'GET')
{{binary}} -tc len(body)>0
@@ -65,7 +65,7 @@
{{binary}} -w workflows
{{binary}} -w workflows -author geeknik,pdteam
{{binary}} -w workflows -severity high,critical
{{binary}} -w workflows -author geeknik,pdteam -severity high,critical
{{binary}} -w workflows -author geeknik,pdteam -severity high,critical
# Input Types
# http protocol
+1 -3
View File
@@ -23,9 +23,7 @@ func main() {
if err != nil {
log.Fatalf("Could not create file: %s\n", err)
}
defer func() {
_ = file.Close()
}()
defer file.Close()
err = filepath.WalkDir(templatesDirectory, func(path string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() {
-3
View File
@@ -24,9 +24,6 @@ var codeTestCases = []TestCaseInfo{
{Path: "protocols/code/py-interactsh.yaml", TestCase: &codeSnippet{}, DisableOn: isCodeDisabled},
{Path: "protocols/code/ps1-snippet.yaml", TestCase: &codeSnippet{}, DisableOn: func() bool { return !osutils.IsWindows() || isCodeDisabled() }},
{Path: "protocols/code/pre-condition.yaml", TestCase: &codePreCondition{}, DisableOn: isCodeDisabled},
{Path: "protocols/code/sh-virtual.yaml", TestCase: &codeSnippet{}, DisableOn: func() bool { return !osutils.IsLinux() || isCodeDisabled() }},
{Path: "protocols/code/py-virtual.yaml", TestCase: &codeSnippet{}, DisableOn: func() bool { return !osutils.IsLinux() || isCodeDisabled() }},
{Path: "protocols/code/pwsh-echo.yaml", TestCase: &codeSnippet{}, DisableOn: func() bool { return isCodeDisabled() }},
}
const (
+1 -3
View File
@@ -18,9 +18,7 @@ func (h *customConfigDirTest) Execute(filePath string) error {
if err != nil {
return err
}
defer func() {
_ = os.RemoveAll(customTempDirectory)
}()
defer os.RemoveAll(customTempDirectory)
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, []string{"NUCLEI_CONFIG_DIR=" + customTempDirectory}, "-t", filePath, "-u", "8x8exch02.8x8.com")
if err != nil {
return err
+2 -2
View File
@@ -21,7 +21,7 @@ type dslVersionWarning struct{}
func (d *dslVersionWarning) Execute(templatePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "DSL version parsing warning test")
fmt.Fprintf(w, "DSL version parsing warning test")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -37,7 +37,7 @@ type dslShowVersionWarning struct{}
func (d *dslShowVersionWarning) Execute(templatePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "DSL version parsing warning test")
fmt.Fprintf(w, "DSL version parsing warning test")
})
ts := httptest.NewServer(router)
defer ts.Close()
-104
View File
@@ -1,104 +0,0 @@
package main
import (
"context"
"fmt"
"log"
"time"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/exporters/mongo"
"github.com/testcontainers/testcontainers-go"
mongocontainer "github.com/testcontainers/testcontainers-go/modules/mongodb"
osutil "github.com/projectdiscovery/utils/os"
mongoclient "go.mongodb.org/mongo-driver/mongo"
mongooptions "go.mongodb.org/mongo-driver/mongo/options"
)
const (
dbName = "test"
dbImage = "mongo:8"
)
var exportersTestCases = []TestCaseInfo{
{Path: "exporters/mongo", TestCase: &mongoExporter{}, DisableOn: func() bool {
return osutil.IsWindows() || osutil.IsOSX()
}},
}
type mongoExporter struct{}
func (m *mongoExporter) Execute(filepath string) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
// Start a MongoDB container
mongodbContainer, err := mongocontainer.Run(ctx, dbImage)
defer func() {
if err := testcontainers.TerminateContainer(mongodbContainer); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()
if err != nil {
return fmt.Errorf("failed to start container: %w", err)
}
connString, err := mongodbContainer.ConnectionString(ctx)
if err != nil {
return fmt.Errorf("failed to get connection string for MongoDB container: %s", err)
}
connString = connString + dbName
// Create a MongoDB exporter and write a test result to the database
opts := mongo.Options{
ConnectionString: connString,
CollectionName: "test",
BatchSize: 1, // Ensure we write the result immediately
}
exporter, err := mongo.New(&opts)
if err != nil {
return fmt.Errorf("failed to create MongoDB exporter: %s", err)
}
defer func() {
if err := exporter.Close(); err != nil {
fmt.Printf("failed to close exporter: %s\n", err)
}
}()
res := &output.ResultEvent{
Request: "test request",
Response: "test response",
}
err = exporter.Export(res)
if err != nil {
return fmt.Errorf("failed to export result event to MongoDB: %s", err)
}
// Verify that the result was written to the database
clientOptions := mongooptions.Client().ApplyURI(connString)
client, err := mongoclient.Connect(ctx, clientOptions)
if err != nil {
return fmt.Errorf("error creating MongoDB client: %s", err)
}
defer func() {
if err := client.Disconnect(ctx); err != nil {
fmt.Printf("failed to disconnect from MongoDB: %s\n", err)
}
}()
collection := client.Database(dbName).Collection(opts.CollectionName)
var actualRes output.ResultEvent
err = collection.FindOne(ctx, map[string]interface{}{"request": res.Request}).Decode(&actualRes)
if err != nil {
return fmt.Errorf("failed to find document in MongoDB: %s", err)
}
if actualRes.Request != res.Request || actualRes.Response != res.Response {
return fmt.Errorf("exported result does not match expected result: got %v, want %v", actualRes, res)
}
return nil
}
+1 -1
View File
@@ -49,7 +49,7 @@ func (t *iterateValuesFlow) Execute(filePath string) error {
}
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, testemails)
_, _ = w.Write([]byte(fmt.Sprint(testemails)))
})
router.GET("/user/"+getBase64(testemails[0]), func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.WriteHeader(http.StatusOK)
+5 -5
View File
@@ -55,7 +55,7 @@ func (h *httpFuzzQuery) Execute(filePath string) error {
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.Header().Set("Content-Type", "text/html")
value := r.URL.Query().Get("id")
_, _ = fmt.Fprintf(w, "This is test matcher text: %v", value)
fmt.Fprintf(w, "This is test matcher text: %v", value)
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -75,7 +75,7 @@ func (h *fuzzModeOverride) Execute(filePath string) error {
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.Header().Set("Content-Type", "text/html")
value := r.URL.Query().Get("id")
_, _ = fmt.Fprintf(w, "This is test matcher text: %v", value)
fmt.Fprintf(w, "This is test matcher text: %v", value)
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -120,7 +120,7 @@ func (h *fuzzTypeOverride) Execute(filePath string) error {
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.Header().Set("Content-Type", "text/html")
value := r.URL.Query().Get("id")
_, _ = fmt.Fprintf(w, "This is test matcher text: %v", value)
fmt.Fprintf(w, "This is test matcher text: %v", value)
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -164,7 +164,7 @@ func (h *HeadlessFuzzingQuery) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
resp := fmt.Sprintf("<html><body>%s</body></html>", r.URL.Query().Get("url"))
_, _ = fmt.Fprint(w, resp)
fmt.Fprint(w, resp)
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -190,7 +190,7 @@ func (h *fuzzMultipleMode) Execute(filePath string) error {
}
w.Header().Set("Content-Type", "text/html")
resp := fmt.Sprintf("<html><body><h1>This is multi-mode fuzzing test: %v <h1></body></html>", xClientId)
_, _ = fmt.Fprint(w, resp)
fmt.Fprint(w, resp)
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
+4 -5
View File
@@ -82,15 +82,14 @@ func (h *clientCertificate) Execute(filePath string) error {
return
}
_, _ = fmt.Fprintf(w, "Hello, %s!\n", r.TLS.PeerCertificates[0].Subject)
fmt.Fprintf(w, "Hello, %s!\n", r.TLS.PeerCertificates[0].Subject)
})
_ = os.WriteFile("server.crt", []byte(serverCRT), permissionutil.ConfigFilePermission)
_ = os.WriteFile("server.key", []byte(serverKey), permissionutil.ConfigFilePermission)
defer func() {
_ = os.Remove("server.crt")
_ = os.Remove("server.key")
}()
defer os.Remove("server.crt")
defer os.Remove("server.key")
serverCert, _ := tls.LoadX509KeyPair("server.crt", "server.key")
certPool := x509.NewCertPool()
+2 -6
View File
@@ -178,9 +178,7 @@ func (h *headlessFileUpload) Execute(filePath string) error {
return
}
defer func() {
_ = file.Close()
}()
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
@@ -237,9 +235,7 @@ func (h *headlessFileUploadNegative) Execute(filePath string) error {
return
}
defer func() {
_ = file.Close()
}()
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
+56 -128
View File
@@ -19,7 +19,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
logutil "github.com/projectdiscovery/utils/log"
sliceutil "github.com/projectdiscovery/utils/slice"
stringsutil "github.com/projectdiscovery/utils/strings"
@@ -62,11 +62,9 @@ var httpTestcases = []TestCaseInfo{
{Path: "protocols/http/dsl-functions.yaml", TestCase: &httpDSLFunctions{}},
{Path: "protocols/http/race-simple.yaml", TestCase: &httpRaceSimple{}},
{Path: "protocols/http/race-multiple.yaml", TestCase: &httpRaceMultiple{}},
{Path: "protocols/http/race-with-variables.yaml", TestCase: &httpRaceWithVariables{}},
{Path: "protocols/http/stop-at-first-match.yaml", TestCase: &httpStopAtFirstMatch{}},
{Path: "protocols/http/stop-at-first-match-with-extractors.yaml", TestCase: &httpStopAtFirstMatchWithExtractors{}},
{Path: "protocols/http/variables.yaml", TestCase: &httpVariables{}},
{Path: "protocols/http/variables-threads-previous.yaml", TestCase: &httpVariablesThreadsPrevious{}},
{Path: "protocols/http/variable-dsl-function.yaml", TestCase: &httpVariableDSLFunction{}},
{Path: "protocols/http/get-override-sni.yaml", TestCase: &httpSniAnnotation{}},
{Path: "protocols/http/get-sni.yaml", TestCase: &customCLISNI{}},
@@ -79,7 +77,6 @@ var httpTestcases = []TestCaseInfo{
{Path: "protocols/http/cl-body-without-header.yaml", TestCase: &httpCLBodyWithoutHeader{}},
{Path: "protocols/http/cl-body-with-header.yaml", TestCase: &httpCLBodyWithHeader{}},
{Path: "protocols/http/cli-with-constants.yaml", TestCase: &ConstantWithCliVar{}},
{Path: "protocols/http/constants-with-threads.yaml", TestCase: &constantsWithThreads{}},
{Path: "protocols/http/matcher-status.yaml", TestCase: &matcherStatusTest{}},
{Path: "protocols/http/disable-path-automerge.yaml", TestCase: &httpDisablePathAutomerge{}},
{Path: "protocols/http/http-preprocessor.yaml", TestCase: &httpPreprocessor{}},
@@ -111,7 +108,7 @@ func (h *httpMatcherExtractorDynamicExtractor) Execute(filePath string) error {
<a href="/domains">Domains</a>
</body>
</html>`
_, _ = fmt.Fprint(w, html)
fmt.Fprint(w, html)
})
router.GET("/domains", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
html := `<!DOCTYPE html>
@@ -124,7 +121,7 @@ func (h *httpMatcherExtractorDynamicExtractor) Execute(filePath string) error {
</body>
</html>
`
_, _ = fmt.Fprint(w, html)
fmt.Fprint(w, html)
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -146,7 +143,7 @@ func (h *httpInteractshRequest) Execute(filePath string) error {
value := r.Header.Get("url")
if value != "" {
if resp, _ := retryablehttp.DefaultClient().Get(value); resp != nil {
_ = resp.Body.Close()
resp.Body.Close()
}
}
})
@@ -199,7 +196,7 @@ func (d *httpDefaultMatcherCondition) Execute(filePath string) error {
return err
}
if routerErr != nil {
return errkit.Wrap(routerErr, "failed to send http request to interactsh server")
return errorutil.NewWithErr(routerErr).Msgf("failed to send http request to interactsh server")
}
if err := expectResultsCount(results, 1); err != nil {
return err
@@ -216,7 +213,7 @@ func (h *httpInteractshStopAtFirstMatchRequest) Execute(filePath string) error {
value := r.Header.Get("url")
if value != "" {
if resp, _ := retryablehttp.DefaultClient().Get(value); resp != nil {
_ = resp.Body.Close()
resp.Body.Close()
}
}
})
@@ -238,7 +235,7 @@ func (h *httpGetHeaders) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
ts := httptest.NewServer(router)
@@ -259,7 +256,7 @@ func (h *httpGetQueryString) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
if strings.EqualFold(r.URL.Query().Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test querystring matcher text")
fmt.Fprintf(w, "This is test querystring matcher text")
}
})
ts := httptest.NewServer(router)
@@ -282,7 +279,7 @@ func (h *httpGetRedirects) Execute(filePath string) error {
http.Redirect(w, r, "/redirected", http.StatusFound)
})
router.GET("/redirected", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test redirects matcher text")
fmt.Fprintf(w, "This is test redirects matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -332,7 +329,7 @@ func (h *httpDisableRedirects) Execute(filePath string) error {
http.Redirect(w, r, "/redirected", http.StatusMovedPermanently)
})
router.GET("/redirected", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test redirects matcher text")
fmt.Fprintf(w, "This is test redirects matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -351,7 +348,7 @@ type httpGet struct{}
func (h *httpGet) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -370,7 +367,7 @@ type httpDSLVariable struct{}
func (h *httpDSLVariable) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -453,7 +450,7 @@ func (h *httpPostBody) Execute(filePath string) error {
return
}
if strings.EqualFold(r.Form.Get("username"), "test") && strings.EqualFold(r.Form.Get("password"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test post-body matcher text")
fmt.Fprintf(w, "This is test post-body matcher text")
}
})
ts := httptest.NewServer(router)
@@ -488,7 +485,7 @@ func (h *httpPostJSONBody) Execute(filePath string) error {
return
}
if strings.EqualFold(obj.Username, "test") && strings.EqualFold(obj.Password, "nuclei") {
_, _ = fmt.Fprintf(w, "This is test post-json-body matcher text")
fmt.Fprintf(w, "This is test post-json-body matcher text")
}
})
ts := httptest.NewServer(router)
@@ -528,7 +525,7 @@ func (h *httpPostMultipartBody) Execute(filePath string) error {
return
}
if strings.EqualFold(password[0], "nuclei") && strings.EqualFold(file[0].Filename, "username") {
_, _ = fmt.Fprintf(w, "This is test post-multipart matcher text")
fmt.Fprintf(w, "This is test post-multipart matcher text")
}
})
ts := httptest.NewServer(router)
@@ -558,12 +555,12 @@ func (h *httpRawDynamicExtractor) Execute(filePath string) error {
return
}
if strings.EqualFold(r.Form.Get("testing"), "parameter") {
_, _ = fmt.Fprintf(w, "Token: 'nuclei'")
fmt.Fprintf(w, "Token: 'nuclei'")
}
})
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
if strings.EqualFold(r.URL.Query().Get("username"), "nuclei") {
_, _ = fmt.Fprintf(w, "Test is test-dynamic-extractor-raw matcher text")
fmt.Fprintf(w, "Test is test-dynamic-extractor-raw matcher text")
}
})
ts := httptest.NewServer(router)
@@ -587,7 +584,7 @@ func (h *httpRawGetQuery) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
if strings.EqualFold(r.URL.Query().Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "Test is test raw-get-query-matcher text")
fmt.Fprintf(w, "Test is test raw-get-query-matcher text")
}
})
ts := httptest.NewServer(router)
@@ -607,7 +604,7 @@ type httpRawGet struct{}
func (h *httpRawGet) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "Test is test raw-get-matcher text")
fmt.Fprintf(w, "Test is test raw-get-matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -631,12 +628,12 @@ func (h *httpRawWithParams) Execute(filePath string) error {
// we intentionally use params["test"] instead of params.Get("test") to test the case where
// there are multiple parameters with the same name
if !reflect.DeepEqual(params["key1"], []string{"value1"}) {
errx = errkit.Append(errx, errkit.New("key1 not found in params", "expected", []string{"value1"}, "got", params["key1"]))
errx = errorutil.WrapfWithNil(errx, "expected %v, got %v", []string{"value1"}, params["key1"])
}
if !reflect.DeepEqual(params["key2"], []string{"value2"}) {
errx = errkit.Append(errx, errkit.New("key2 not found in params", "expected", []string{"value2"}, "got", params["key2"]))
errx = errorutil.WrapfWithNil(errx, "expected %v, got %v", []string{"value2"}, params["key2"])
}
_, _ = fmt.Fprintf(w, "Test is test raw-params-matcher text")
fmt.Fprintf(w, "Test is test raw-params-matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -688,11 +685,11 @@ func (h *httpRawPayload) Execute(filePath string) error {
routerErr = err
return
}
if !strings.EqualFold(r.Header.Get("another_header"), "bnVjbGVp") && !strings.EqualFold(r.Header.Get("another_header"), "Z3Vlc3Q=") {
if !(strings.EqualFold(r.Header.Get("another_header"), "bnVjbGVp") || strings.EqualFold(r.Header.Get("another_header"), "Z3Vlc3Q=")) {
return
}
if strings.EqualFold(r.Form.Get("username"), "test") && (strings.EqualFold(r.Form.Get("password"), "nuclei") || strings.EqualFold(r.Form.Get("password"), "guest")) {
_, _ = fmt.Fprintf(w, "Test is raw-payload matcher text")
fmt.Fprintf(w, "Test is raw-payload matcher text")
}
})
ts := httptest.NewServer(router)
@@ -722,7 +719,7 @@ func (h *httpRawPostBody) Execute(filePath string) error {
return
}
if strings.EqualFold(r.Form.Get("username"), "test") && strings.EqualFold(r.Form.Get("password"), "nuclei") {
_, _ = fmt.Fprintf(w, "Test is test raw-post-body-matcher text")
fmt.Fprintf(w, "Test is test raw-post-body-matcher text")
}
})
ts := httptest.NewServer(router)
@@ -832,7 +829,10 @@ func (h *httpPaths) Execute(filepath string) error {
}
if len(expected) > len(actual) {
actualValuesIndex := max(len(actual)-1, 0)
actualValuesIndex := len(actual) - 1
if actualValuesIndex < 0 {
actualValuesIndex = 0
}
return fmt.Errorf("missing values : %v", expected[actualValuesIndex:])
} else if len(expected) < len(actual) {
return fmt.Errorf("unexpected values : %v", actual[len(expected)-1:])
@@ -872,7 +872,7 @@ func (h *httpRawCookieReuse) Execute(filePath string) error {
}
if strings.EqualFold(cookie.Value, "test") {
_, _ = fmt.Fprintf(w, "Test is test-cookie-reuse matcher text")
fmt.Fprintf(w, "Test is test-cookie-reuse matcher text")
}
})
ts := httptest.NewServer(router)
@@ -950,9 +950,7 @@ func (h *httpRequestSelfContained) Execute(filePath string) error {
go func() {
_ = server.ListenAndServe()
}()
defer func() {
_ = server.Close()
}()
defer server.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "", debug, "-esc")
if err != nil {
@@ -974,10 +972,10 @@ func (h *httpRequestSelfContainedWithParams) Execute(filePath string) error {
// we intentionally use params["test"] instead of params.Get("test") to test the case where
// there are multiple parameters with the same name
if !reflect.DeepEqual(params["something"], []string{"here"}) {
errx = errkit.Append(errx, errkit.New("something not found in params", "expected", []string{"here"}, "got", params["something"]))
errx = errorutil.WrapfWithNil(errx, "expected %v, got %v", []string{"here"}, params["something"])
}
if !reflect.DeepEqual(params["key"], []string{"value"}) {
errx = errkit.Append(errx, errkit.New("key not found in params", "expected", []string{"value"}, "got", params["key"]))
errx = errorutil.WrapfWithNil(errx, "expected %v, got %v", []string{"value"}, params["key"])
}
_, _ = w.Write([]byte("This is self-contained response"))
})
@@ -988,9 +986,7 @@ func (h *httpRequestSelfContainedWithParams) Execute(filePath string) error {
go func() {
_ = server.ListenAndServe()
}()
defer func() {
_ = server.Close()
}()
defer server.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "", debug, "-esc")
if err != nil {
@@ -1023,21 +1019,17 @@ func (h *httpRequestSelfContainedFileInput) Execute(filePath string) error {
go func() {
_ = server.ListenAndServe()
}()
defer func() {
_ = server.Close()
}()
defer server.Close()
// create temp file
FileLoc, err := os.CreateTemp("", "self-contained-payload-*.txt")
if err != nil {
return errkit.Wrap(err, "failed to create temp file")
return errorutil.NewWithErr(err).Msgf("failed to create temp file")
}
if _, err := FileLoc.Write([]byte("one\ntwo\n")); err != nil {
return errkit.Wrap(err, "failed to write payload to temp file")
return errorutil.NewWithErr(err).Msgf("failed to write payload to temp file")
}
defer func() {
_ = FileLoc.Close()
}()
defer FileLoc.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "", debug, "-V", "test="+FileLoc.Name(), "-esc")
if err != nil {
@@ -1049,7 +1041,7 @@ func (h *httpRequestSelfContainedFileInput) Execute(filePath string) error {
}
if !sliceutil.ElementsMatch(gotReqToEndpoints, []string{"/one", "/two", "/one", "/two"}) {
return errkit.New("expected requests to be sent to `/one` and `/two` endpoints but were sent to `%v`", gotReqToEndpoints, "filePath", filePath)
return errorutil.NewWithTag(filePath, "expected requests to be sent to `/one` and `/two` endpoints but were sent to `%v`", gotReqToEndpoints)
}
return nil
}
@@ -1060,7 +1052,7 @@ type httpGetCaseInsensitive struct{}
func (h *httpGetCaseInsensitive) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "THIS IS TEST MATCHER TEXT")
fmt.Fprintf(w, "THIS IS TEST MATCHER TEXT")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1079,7 +1071,7 @@ type httpGetCaseInsensitiveCluster struct{}
func (h *httpGetCaseInsensitiveCluster) Execute(filesPath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1156,33 +1148,13 @@ func (h *httpRaceMultiple) Execute(filePath string) error {
return expectResultsCount(results, 5)
}
type httpRaceWithVariables struct{}
// Execute tests that variables and constants are properly resolved in race mode.
func (h *httpRaceWithVariables) Execute(filePath string) error {
router := httprouter.New()
router.GET("/race", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// Echo back the API key header so we can match on it
_, _ = fmt.Fprint(w, r.Header.Get("X-API-Key"))
})
ts := httptest.NewServer(router)
defer ts.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug)
if err != nil {
return err
}
return expectResultsCount(results, 3)
}
type httpStopAtFirstMatch struct{}
// Execute executes a test case and returns an error if occurred
func (h *httpStopAtFirstMatch) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test")
fmt.Fprintf(w, "This is test")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1201,7 +1173,7 @@ type httpStopAtFirstMatchWithExtractors struct{}
func (h *httpStopAtFirstMatchWithExtractors) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test")
fmt.Fprintf(w, "This is test")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1220,7 +1192,7 @@ type httpVariables struct{}
func (h *httpVariables) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "%s\n%s\n%s", r.Header.Get("Test"), r.Header.Get("Another"), r.Header.Get("Email"))
fmt.Fprintf(w, "%s\n%s\n%s", r.Header.Get("Test"), r.Header.Get("Another"), r.Header.Get("Email"))
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1243,30 +1215,6 @@ func (h *httpVariables) Execute(filePath string) error {
return expectResultsCount(results, 0)
}
type httpVariablesThreadsPrevious struct{}
// Execute tests that variables can reference data extracted from previous requests
// when using threads mode (parallel execution).
func (h *httpVariablesThreadsPrevious) Execute(filePath string) error {
router := httprouter.New()
router.GET("/login", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprint(w, "token=secret123")
})
router.GET("/api", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
// Echo back the Authorization header so we can match on it
_, _ = fmt.Fprint(w, r.Header.Get("Authorization"))
})
ts := httptest.NewServer(router)
defer ts.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
type httpVariableDSLFunction struct{}
// Execute executes a test case and returns an error if occurred
@@ -1346,7 +1294,7 @@ func (h *httpRedirectMatchURL) Execute(filePath string) error {
_, _ = w.Write([]byte("This is test redirects matcher text"))
})
router.GET("/redirected", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test redirects matcher text")
fmt.Fprintf(w, "This is test redirects matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -1394,7 +1342,7 @@ func (h *annotationTimeout) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
time.Sleep(4 * time.Second)
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -1414,7 +1362,7 @@ func (h *customAttackType) Execute(filePath string) error {
got := []string{}
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
got = append(got, r.URL.RawQuery)
_, _ = fmt.Fprintf(w, "This is test custom payload")
fmt.Fprintf(w, "This is test custom payload")
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -1462,7 +1410,7 @@ func (h *httpCLBodyWithoutHeader) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.Header()["Content-Length"] = []string{"-1"}
_, _ = fmt.Fprintf(w, "this is a test")
fmt.Fprintf(w, "this is a test")
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -1482,7 +1430,7 @@ func (h *httpCLBodyWithHeader) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.Header()["Content-Length"] = []string{"50000"}
_, _ = fmt.Fprintf(w, "this is a test")
fmt.Fprintf(w, "this is a test")
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -1501,7 +1449,7 @@ type ConstantWithCliVar struct{}
func (h *ConstantWithCliVar) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprint(w, r.URL.Query().Get("p"))
fmt.Fprint(w, r.URL.Query().Get("p"))
})
ts := httptest.NewTLSServer(router)
defer ts.Close()
@@ -1513,26 +1461,6 @@ func (h *ConstantWithCliVar) Execute(filePath string) error {
return expectResultsCount(got, 1)
}
type constantsWithThreads struct{}
// Execute tests that constants are properly resolved when using threads mode.
func (h *constantsWithThreads) Execute(filePath string) error {
router := httprouter.New()
router.GET("/api/:version", func(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
// Echo back the API key header and version so we can match on them
_, _ = fmt.Fprintf(w, "%s %s", r.Header.Get("X-API-Key"), p.ByName("version"))
})
ts := httptest.NewServer(router)
defer ts.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
type matcherStatusTest struct{}
// Execute executes a test case and returns an error if occurred
@@ -1558,10 +1486,10 @@ type httpDisablePathAutomerge struct{}
func (h *httpDisablePathAutomerge) Execute(filePath string) error {
router := httprouter.New()
router.GET("/api/v1/test", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprint(w, r.URL.Query().Get("id"))
fmt.Fprint(w, r.URL.Query().Get("id"))
})
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprint(w, "empty path in raw request")
fmt.Fprint(w, "empty path in raw request")
})
ts := httptest.NewServer(router)
@@ -1595,10 +1523,10 @@ func (h *httpPreprocessor) Execute(filePath string) error {
value := r.URL.RequestURI()
if re.MatchString(value) {
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, "ok")
fmt.Fprint(w, "ok")
} else {
w.WriteHeader(http.StatusBadRequest)
_, _ = fmt.Fprint(w, "not ok")
fmt.Fprint(w, "not ok")
}
})
ts := httptest.NewServer(router)
@@ -1619,11 +1547,11 @@ func (h *httpMultiRequest) Execute(filePath string) error {
router := httprouter.New()
router.GET("/ping", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, "ping")
fmt.Fprint(w, "ping")
})
router.GET("/pong", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, "pong")
fmt.Fprint(w, "pong")
})
ts := httptest.NewServer(router)
defer ts.Close()
+5 -53
View File
@@ -4,9 +4,7 @@ import (
"flag"
"fmt"
"os"
"regexp"
"runtime"
"slices"
"strings"
"github.com/kitabisa/go-ci"
@@ -25,7 +23,7 @@ type TestCaseInfo struct {
}
var (
debug = isDebugMode()
debug = os.Getenv("DEBUG") == "true"
customTests = os.Getenv("TESTS")
protocol = os.Getenv("PROTO")
@@ -47,7 +45,6 @@ var (
"library": libraryTestcases,
"templatesPath": templatesPathTestCases,
"templatesDir": templatesDirTestCases,
"env_vars": templatesDirEnvTestCases,
"file": fileTestcases,
"offlineHttp": offlineHttpTestcases,
"customConfigDir": customConfigDirTestCases,
@@ -59,9 +56,7 @@ var (
"flow": flowTestcases,
"javascript": jsTestcases,
"matcher-status": matcherStatusTestcases,
"exporters": exportersTestCases,
}
// flakyTests are run with a retry count of 3
flakyTests = map[string]bool{
"protocols/http/self-contained-file-input.yaml": true,
@@ -92,12 +87,9 @@ func main() {
}
// start fuzz playground server
defer fuzzplayground.Cleanup()
server := fuzzplayground.GetPlaygroundServer()
defer func() {
fuzzplayground.Cleanup()
_ = server.Close()
}()
defer server.Close()
go func() {
if err := server.Start("localhost:8082"); err != nil {
if !strings.Contains(err.Error(), "Server closed") {
@@ -107,6 +99,7 @@ func main() {
}()
customTestsList := normalizeSplit(customTests)
failedTestTemplatePaths := runTests(customTestsList)
if len(failedTestTemplatePaths) > 0 {
@@ -133,27 +126,6 @@ func main() {
}
}
// isDebugMode checks if debug mode is enabled via any of the supported debug
// environment variables.
func isDebugMode() bool {
debugEnvVars := []string{
"DEBUG",
"ACTIONS_RUNNER_DEBUG", // GitHub Actions runner debug
// Add more debug environment variables here as needed
}
truthyValues := []string{"true", "1", "yes", "on", "enabled"}
for _, envVar := range debugEnvVars {
envValue := strings.ToLower(strings.TrimSpace(os.Getenv(envVar)))
if slices.Contains(truthyValues, envValue) {
return true
}
}
return false
}
// execute a testcase with retry and consider best of N
// intended for flaky tests like interactsh
func executeWithRetry(testCase testutils.TestCase, templatePath string, retryCount int) (string, error) {
@@ -236,7 +208,7 @@ func execute(testCase testutils.TestCase, templatePath string) (string, error) {
}
func expectResultsCount(results []string, expectedNumbers ...int) error {
results = filterLines(results)
results = filterHeadlessLogs(results)
match := sliceutil.Contains(expectedNumbers, len(results))
if !match {
return fmt.Errorf("incorrect number of results: %d (actual) vs %v (expected) \nResults:\n\t%s\n", len(results), expectedNumbers, strings.Join(results, "\n\t")) // nolint:all
@@ -250,13 +222,6 @@ func normalizeSplit(str string) []string {
})
}
// filterLines applies all filtering functions to the results
func filterLines(results []string) []string {
results = filterHeadlessLogs(results)
results = filterUnsignedTemplatesWarnings(results)
return results
}
// if chromium is not installed go-rod installs it in .cache directory
// this function filters out the logs from download and installation
func filterHeadlessLogs(results []string) []string {
@@ -270,16 +235,3 @@ func filterHeadlessLogs(results []string) []string {
}
return filtered
}
// filterUnsignedTemplatesWarnings filters out warning messages about unsigned templates
func filterUnsignedTemplatesWarnings(results []string) []string {
filtered := []string{}
unsignedTemplatesRegex := regexp.MustCompile(`Loading \d+ unsigned templates for scan\. Use with caution\.`)
for _, result := range results {
if unsignedTemplatesRegex.MatchString(result) {
continue
}
filtered = append(filtered, result)
}
return filtered
}
+4 -238
View File
@@ -15,23 +15,13 @@ var jsTestcases = []TestCaseInfo{
{Path: "protocols/javascript/ssh-server-fingerprint.yaml", TestCase: &javascriptSSHServerFingerprint{}, DisableOn: func() bool { return osutils.IsWindows() || osutils.IsOSX() }},
{Path: "protocols/javascript/net-multi-step.yaml", TestCase: &networkMultiStep{}},
{Path: "protocols/javascript/net-https.yaml", TestCase: &javascriptNetHttps{}},
{Path: "protocols/javascript/oracle-auth-test.yaml", TestCase: &javascriptOracleAuthTest{}, DisableOn: func() bool { return osutils.IsWindows() || osutils.IsOSX() }},
{Path: "protocols/javascript/vnc-pass-brute.yaml", TestCase: &javascriptVncPassBrute{}},
{Path: "protocols/javascript/postgres-pass-brute.yaml", TestCase: &javascriptPostgresPassBrute{}, DisableOn: func() bool { return osutils.IsWindows() || osutils.IsOSX() }},
{Path: "protocols/javascript/mysql-connect.yaml", TestCase: &javascriptMySQLConnect{}, DisableOn: func() bool { return osutils.IsWindows() || osutils.IsOSX() }},
{Path: "protocols/javascript/multi-ports.yaml", TestCase: &javascriptMultiPortsSSH{}},
{Path: "protocols/javascript/no-port-args.yaml", TestCase: &javascriptNoPortArgs{}},
}
var (
redisResource *dockertest.Resource
sshResource *dockertest.Resource
oracleResource *dockertest.Resource
vncResource *dockertest.Resource
postgresResource *dockertest.Resource
mysqlResource *dockertest.Resource
pool *dockertest.Pool
defaultRetry = 3
redisResource *dockertest.Resource
sshResource *dockertest.Resource
pool *dockertest.Pool
defaultRetry = 3
)
type javascriptNetHttps struct{}
@@ -108,156 +98,6 @@ func (j *javascriptSSHServerFingerprint) Execute(filePath string) error {
return multierr.Combine(errs...)
}
type javascriptOracleAuthTest struct{}
func (j *javascriptOracleAuthTest) Execute(filePath string) error {
if oracleResource == nil || pool == nil {
// skip test as oracle is not running
return nil
}
tempPort := oracleResource.GetPort("1521/tcp")
finalURL := "localhost:" + tempPort
defer purge(oracleResource)
errs := []error{}
for i := 0; i < defaultRetry; i++ {
results := []string{}
var err error
_ = pool.Retry(func() error {
//let ssh server start
time.Sleep(3 * time.Second)
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, finalURL, debug)
return nil
})
if err != nil {
return err
}
if err := expectResultsCount(results, 1); err == nil {
return nil
} else {
errs = append(errs, err)
}
}
return multierr.Combine(errs...)
}
type javascriptVncPassBrute struct{}
func (j *javascriptVncPassBrute) Execute(filePath string) error {
if vncResource == nil || pool == nil {
// skip test as vnc is not running
return nil
}
tempPort := vncResource.GetPort("5900/tcp")
finalURL := "localhost:" + tempPort
defer purge(vncResource)
errs := []error{}
for i := 0; i < defaultRetry; i++ {
results := []string{}
var err error
_ = pool.Retry(func() error {
//let ssh server start
time.Sleep(3 * time.Second)
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, finalURL, debug)
return nil
})
if err != nil {
return err
}
if err := expectResultsCount(results, 1); err == nil {
return nil
} else {
errs = append(errs, err)
}
}
return multierr.Combine(errs...)
}
type javascriptPostgresPassBrute struct{}
func (j *javascriptPostgresPassBrute) Execute(filePath string) error {
if postgresResource == nil || pool == nil {
// skip test as postgres is not running
return nil
}
tempPort := postgresResource.GetPort("5432/tcp")
finalURL := "localhost:" + tempPort
defer purge(postgresResource)
errs := []error{}
for i := 0; i < defaultRetry; i++ {
results := []string{}
var err error
_ = pool.Retry(func() error {
//let postgres server start
time.Sleep(3 * time.Second)
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, finalURL, debug)
return nil
})
if err != nil {
return err
}
if err := expectResultsCount(results, 1); err == nil {
return nil
} else {
errs = append(errs, err)
}
}
return multierr.Combine(errs...)
}
type javascriptMySQLConnect struct{}
func (j *javascriptMySQLConnect) Execute(filePath string) error {
if mysqlResource == nil || pool == nil {
// skip test as mysql is not running
return nil
}
tempPort := mysqlResource.GetPort("3306/tcp")
finalURL := "localhost:" + tempPort
defer purge(mysqlResource)
errs := []error{}
for i := 0; i < defaultRetry; i++ {
results := []string{}
var err error
_ = pool.Retry(func() error {
//let mysql server start
time.Sleep(5 * time.Second)
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, finalURL, debug)
return nil
})
if err != nil {
return err
}
if err := expectResultsCount(results, 1); err == nil {
return nil
} else {
errs = append(errs, err)
}
}
return multierr.Combine(errs...)
}
type javascriptMultiPortsSSH struct{}
func (j *javascriptMultiPortsSSH) Execute(filePath string) error {
// use scanme.sh as target to ensure we match on the 2nd default port 22
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "scanme.sh", debug)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
type javascriptNoPortArgs struct{}
func (j *javascriptNoPortArgs) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "yo.dawg", debug)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// purge any given resource if it is not nil
func purge(resource *dockertest.Resource) {
if resource != nil && pool != nil {
@@ -323,78 +163,4 @@ func init() {
if err := sshResource.Expire(30); err != nil {
log.Printf("Could not expire resource: %s", err)
}
// setup a temporary oracle instance
oracleResource, err = pool.RunWithOptions(&dockertest.RunOptions{
Repository: "gvenzl/oracle-xe",
Tag: "latest",
Env: []string{
"ORACLE_PASSWORD=mysecret",
},
Platform: "linux/amd64",
})
if err != nil {
log.Printf("Could not start Oracle resource: %s", err)
return
}
// by default expire after 30 sec
if err := oracleResource.Expire(30); err != nil {
log.Printf("Could not expire Oracle resource: %s", err)
}
// setup a temporary vnc server
vncResource, err = pool.RunWithOptions(&dockertest.RunOptions{
Repository: "dorowu/ubuntu-desktop-lxde-vnc",
Tag: "latest",
Env: []string{
"VNC_PASSWORD=mysecret",
},
Platform: "linux/amd64",
})
if err != nil {
log.Printf("Could not start resource: %s", err)
return
}
// by default expire after 30 sec
if err := vncResource.Expire(30); err != nil {
log.Printf("Could not expire resource: %s", err)
}
// setup a temporary postgres instance
postgresResource, err = pool.RunWithOptions(&dockertest.RunOptions{
Repository: "postgres",
Tag: "latest",
Env: []string{
"POSTGRES_PASSWORD=postgres",
"POSTGRES_USER=postgres",
},
Platform: "linux/amd64",
})
if err != nil {
log.Printf("Could not start postgres resource: %s", err)
return
}
// by default expire after 30 sec
if err := postgresResource.Expire(30); err != nil {
log.Printf("Could not expire postgres resource: %s", err)
}
// setup a temporary mysql instance
mysqlResource, err = pool.RunWithOptions(&dockertest.RunOptions{
Repository: "mysql",
Tag: "latest",
Env: []string{
"MYSQL_ROOT_PASSWORD=secret",
},
Platform: "linux/amd64",
})
if err != nil {
log.Printf("Could not start mysql resource: %s", err)
return
}
// by default expire after 30 sec
if err := mysqlResource.Expire(30); err != nil {
log.Printf("Could not expire mysql resource: %s", err)
}
}
+7 -13
View File
@@ -15,7 +15,6 @@ import (
"github.com/logrusorgru/aurora"
"github.com/pkg/errors"
"github.com/projectdiscovery/goflags"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
@@ -49,9 +48,9 @@ func (h *goIntegrationTest) Execute(templatePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
ts := httptest.NewServer(router)
@@ -69,22 +68,17 @@ func executeNucleiAsLibrary(templatePath, templateURL string) ([]string, error)
cache := hosterrorscache.New(30, hosterrorscache.DefaultMaxHostsCount, nil)
defer cache.Close()
defaultOpts := types.DefaultOptions()
defaultOpts.ExecutionId = "test"
defaultOpts.Logger = gologger.DefaultLogger
mockProgress := &testutils.MockProgressClient{}
reportingClient, err := reporting.New(&reporting.Options{ExecutionId: defaultOpts.ExecutionId}, "", false)
reportingClient, err := reporting.New(&reporting.Options{}, "", false)
if err != nil {
return nil, err
}
defer reportingClient.Close()
defaultOpts := types.DefaultOptions()
_ = protocolstate.Init(defaultOpts)
_ = protocolinit.Init(defaultOpts)
defer protocolstate.Close(defaultOpts.ExecutionId)
defaultOpts.Templates = goflags.StringSlice{templatePath}
defaultOpts.ExcludeTags = config.ReadIgnoreFile().Tags
@@ -106,7 +100,7 @@ func executeNucleiAsLibrary(templatePath, templateURL string) ([]string, error)
ratelimiter := ratelimit.New(context.Background(), 150, time.Second)
defer ratelimiter.Stop()
executerOpts := &protocols.ExecutorOptions{
executerOpts := protocols.ExecutorOptions{
Output: outputWriter,
Options: defaultOpts,
Progress: mockProgress,
@@ -122,7 +116,7 @@ func executeNucleiAsLibrary(templatePath, templateURL string) ([]string, error)
engine := core.New(defaultOpts)
engine.SetExecuterOptions(executerOpts)
workflowLoader, err := parsers.NewLoader(executerOpts)
workflowLoader, err := parsers.NewLoader(&executerOpts)
if err != nil {
log.Fatalf("Could not create workflow loader: %s\n", err)
}
@@ -134,7 +128,7 @@ func executeNucleiAsLibrary(templatePath, templateURL string) ([]string, error)
}
store.Load()
_ = engine.Execute(context.Background(), store.Templates(), provider.NewSimpleInputProviderWithUrls(defaultOpts.ExecutionId, templateURL))
_ = engine.Execute(context.Background(), store.Templates(), provider.NewSimpleInputProviderWithUrls(templateURL))
engine.WorkPool().Wait() // Wait for the scan to finish
return results, nil
+14 -22
View File
@@ -10,7 +10,7 @@ import (
"github.com/julienschmidt/httprouter"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
permissionutil "github.com/projectdiscovery/utils/permission"
)
@@ -31,9 +31,9 @@ func (h *remoteTemplateList) Execute(templateList string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
@@ -55,9 +55,7 @@ func (h *remoteTemplateList) Execute(templateList string) error {
if err != nil {
return err
}
defer func() {
_ = os.Remove("test-config.yaml")
}()
defer os.Remove("test-config.yaml")
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, nil, "-target", ts.URL, "-template-url", ts.URL+"/template_list", "-config", "test-config.yaml")
if err != nil {
@@ -74,9 +72,9 @@ func (h *excludedTemplate) Execute(templateList string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
ts := httptest.NewServer(router)
@@ -97,9 +95,9 @@ func (h *remoteTemplateListNotAllowed) Execute(templateList string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
@@ -132,9 +130,9 @@ func (h *remoteWorkflowList) Execute(workflowList string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
if strings.EqualFold(r.Header.Get("test"), "nuclei") {
_, _ = fmt.Fprintf(w, "This is test headers matcher text")
fmt.Fprintf(w, "This is test headers matcher text")
}
})
@@ -156,9 +154,7 @@ func (h *remoteWorkflowList) Execute(workflowList string) error {
if err != nil {
return err
}
defer func() {
_ = os.Remove("test-config.yaml")
}()
defer os.Remove("test-config.yaml")
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, nil, "-target", ts.URL, "-workflow-url", ts.URL+"/workflow_list", "-config", "test-config.yaml")
if err != nil {
@@ -181,9 +177,7 @@ func (h *nonExistentTemplateList) Execute(nonExistingTemplateList string) error
if err != nil {
return err
}
defer func() {
_ = os.Remove("test-config.yaml")
}()
defer os.Remove("test-config.yaml")
_, err = testutils.RunNucleiBareArgsAndGetResults(debug, nil, "-target", ts.URL, "-template-url", ts.URL+"/404", "-config", "test-config.yaml")
if err == nil {
@@ -206,9 +200,7 @@ func (h *nonExistentWorkflowList) Execute(nonExistingWorkflowList string) error
if err != nil {
return err
}
defer func() {
_ = os.Remove("test-config.yaml")
}()
defer os.Remove("test-config.yaml")
_, err = testutils.RunNucleiBareArgsAndGetResults(debug, nil, "-target", ts.URL, "-workflow-url", ts.URL+"/404", "-config", "test-config.yaml")
if err == nil {
@@ -223,7 +215,7 @@ type loadTemplateWithID struct{}
func (h *loadTemplateWithID) Execute(nooop string) error {
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, nil, "-target", "scanme.sh", "-id", "self-signed-ssl")
if err != nil {
return errkit.Wrap(err, "failed to load template with id")
return errorutil.NewWithErr(err).Msgf("failed to load template with id")
}
return expectResultsCount(results, 1)
}
+8 -20
View File
@@ -33,9 +33,7 @@ func (h *networkBasic) Execute(filePath string) error {
var routerErr error
ts := testutils.NewTCPServer(nil, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data, err := reader.ConnReadNWithTimeout(conn, 4, time.Duration(5)*time.Second)
if err != nil {
@@ -52,11 +50,11 @@ func (h *networkBasic) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug)
if err != nil {
_, _ = fmt.Fprintf(os.Stderr, "Could not run nuclei: %s\n", err)
fmt.Fprintf(os.Stderr, "Could not run nuclei: %s\n", err)
return err
}
if routerErr != nil {
_, _ = fmt.Fprintf(os.Stderr, "routerErr: %s\n", routerErr)
fmt.Fprintf(os.Stderr, "routerErr: %s\n", routerErr)
return routerErr
}
@@ -70,9 +68,7 @@ func (h *networkMultiStep) Execute(filePath string) error {
var routerErr error
ts := testutils.NewTCPServer(nil, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data, err := reader.ConnReadNWithTimeout(conn, 5, time.Duration(5)*time.Second)
if err != nil {
@@ -118,9 +114,7 @@ type networkRequestSelContained struct{}
// Execute executes a test case and returns an error if occurred
func (h *networkRequestSelContained) Execute(filePath string) error {
ts := testutils.NewTCPServer(nil, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
_, _ = conn.Write([]byte("Authentication successful"))
})
@@ -140,9 +134,7 @@ func (h *networkVariables) Execute(filePath string) error {
var routerErr error
ts := testutils.NewTCPServer(nil, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data, err := reader.ConnReadNWithTimeout(conn, 4, time.Duration(5)*time.Second)
if err != nil {
@@ -170,9 +162,7 @@ type networkPort struct{}
func (n *networkPort) Execute(filePath string) error {
ts := testutils.NewTCPServer(nil, 23846, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data, err := reader.ConnReadNWithTimeout(conn, 4, time.Duration(5)*time.Second)
if err != nil {
@@ -205,9 +195,7 @@ func (n *networkPort) Execute(filePath string) error {
// this is positive test case where we expect port to be overridden and 34567 to be used
ts2 := testutils.NewTCPServer(nil, 34567, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data, err := reader.ConnReadNWithTimeout(conn, 4, time.Duration(5)*time.Second)
if err != nil {
+7 -7
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
)
var profileLoaderTestcases = []TestCaseInfo{
@@ -16,9 +16,9 @@ var profileLoaderTestcases = []TestCaseInfo{
type profileLoaderByRelFile struct{}
func (h *profileLoaderByRelFile) Execute(testName string) error {
results, err := testutils.RunNucleiWithArgsAndGetResults(debug, "-tl", "-tp", "cloud.yml")
results, err := testutils.RunNucleiWithArgsAndGetResults(false, "-tl", "-tp", "cloud.yml")
if err != nil {
return errkit.Wrap(err, "failed to load template with id")
return errorutil.NewWithErr(err).Msgf("failed to load template with id")
}
if len(results) <= 10 {
return fmt.Errorf("incorrect result: expected more results than %d, got %v", 10, len(results))
@@ -29,9 +29,9 @@ func (h *profileLoaderByRelFile) Execute(testName string) error {
type profileLoaderById struct{}
func (h *profileLoaderById) Execute(testName string) error {
results, err := testutils.RunNucleiWithArgsAndGetResults(debug, "-tl", "-tp", "cloud")
results, err := testutils.RunNucleiWithArgsAndGetResults(false, "-tl", "-tp", "cloud")
if err != nil {
return errkit.Wrap(err, "failed to load template with id")
return errorutil.NewWithErr(err).Msgf("failed to load template with id")
}
if len(results) <= 10 {
return fmt.Errorf("incorrect result: expected more results than %d, got %v", 10, len(results))
@@ -43,9 +43,9 @@ func (h *profileLoaderById) Execute(testName string) error {
type customProfileLoader struct{}
func (h *customProfileLoader) Execute(filepath string) error {
results, err := testutils.RunNucleiWithArgsAndGetResults(debug, "-tl", "-tp", filepath)
results, err := testutils.RunNucleiWithArgsAndGetResults(false, "-tl", "-tp", filepath)
if err != nil {
return errkit.Wrap(err, "failed to load template with id")
return errorutil.NewWithErr(err).Msgf("failed to load template with id")
}
if len(results) < 1 {
return fmt.Errorf("incorrect result: expected more results than %d, got %v", 1, len(results))
+6 -18
View File
@@ -21,9 +21,7 @@ type sslBasic struct{}
// Execute executes a test case and returns an error if occurred
func (h *sslBasic) Execute(filePath string) error {
ts := testutils.NewTCPServer(&tls.Config{}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
@@ -44,9 +42,7 @@ type sslBasicZtls struct{}
// Execute executes a test case and returns an error if occurred
func (h *sslBasicZtls) Execute(filePath string) error {
ts := testutils.NewTCPServer(&tls.Config{}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
@@ -67,9 +63,7 @@ type sslCustomCipher struct{}
// Execute executes a test case and returns an error if occurred
func (h *sslCustomCipher) Execute(filePath string) error {
ts := testutils.NewTCPServer(&tls.Config{CipherSuites: []uint16{tls.TLS_AES_128_GCM_SHA256}}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
@@ -90,9 +84,7 @@ type sslCustomVersion struct{}
// Execute executes a test case and returns an error if occurred
func (h *sslCustomVersion) Execute(filePath string) error {
ts := testutils.NewTCPServer(&tls.Config{MinVersion: tls.VersionTLS12, MaxVersion: tls.VersionTLS12}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
@@ -112,9 +104,7 @@ type sslWithVars struct{}
func (h *sslWithVars) Execute(filePath string) error {
ts := testutils.NewTCPServer(&tls.Config{}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
@@ -138,9 +128,7 @@ func (h *sslMultiReq) Execute(filePath string) error {
MinVersion: tls.VersionSSL30,
MaxVersion: tls.VersionTLS11,
}, defaultStaticPort, func(conn net.Conn) {
defer func() {
_ = conn.Close()
}()
defer conn.Close()
data := make([]byte, 4)
if _, err := conn.Read(data); err != nil {
return
+3 -5
View File
@@ -4,7 +4,7 @@ import (
"os"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
)
var templatesDirTestCases = []TestCaseInfo{
@@ -17,11 +17,9 @@ type templateDirWithTargetTest struct{}
func (h *templateDirWithTargetTest) Execute(filePath string) error {
tempdir, err := os.MkdirTemp("", "nuclei-update-dir-*")
if err != nil {
return errkit.Wrap(err, "failed to create temp dir")
return errorutil.NewWithErr(err).Msgf("failed to create temp dir")
}
defer func() {
_ = os.RemoveAll(tempdir)
}()
defer os.RemoveAll(tempdir)
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "8x8exch02.8x8.com", debug, "-ud", tempdir)
if err != nil {
-217
View File
@@ -1,217 +0,0 @@
package main
import (
"os"
"path/filepath"
osutils "github.com/projectdiscovery/utils/os"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/utils/errkit"
)
// isNotLinux returns true if not running on Linux (used to skip tests on non-Linux OS)
var isNotLinux = func() bool { return !osutils.IsLinux() }
var templatesDirEnvTestCases = []TestCaseInfo{
{Path: "protocols/dns/cname-fingerprint.yaml", TestCase: &templatesDirEnvBasicTest{}, DisableOn: isNotLinux},
{Path: "protocols/dns/cname-fingerprint.yaml", TestCase: &templatesDirEnvAbsolutePathTest{}, DisableOn: isNotLinux},
{Path: "protocols/dns/cname-fingerprint.yaml", TestCase: &templatesDirEnvRelativePathTest{}, DisableOn: isNotLinux},
{Path: "protocols/dns/cname-fingerprint.yaml", TestCase: &templatesDirEnvPrecedenceTest{}, DisableOn: isNotLinux},
{Path: "protocols/dns/cname-fingerprint.yaml", TestCase: &templatesDirEnvCustomTemplatesTest{}, DisableOn: isNotLinux},
}
// copyTemplateToDir copies a template file to a destination directory, preserving the directory structure
func copyTemplateToDir(templatePath, destDir string) error {
// Read the template file
templateData, err := os.ReadFile(templatePath)
if err != nil {
return errkit.Wrap(err, "failed to read template file")
}
// Create the destination path preserving directory structure
destPath := filepath.Join(destDir, templatePath)
destDirPath := filepath.Dir(destPath)
// Create the destination directory if it doesn't exist
if err := os.MkdirAll(destDirPath, 0755); err != nil {
return errkit.Wrap(err, "failed to create destination directory")
}
// Write the template file
if err := os.WriteFile(destPath, templateData, 0644); err != nil {
return errkit.Wrap(err, "failed to write template file")
}
return nil
}
// templatesDirEnvBasicTest tests basic functionality of NUCLEI_TEMPLATES_DIR
type templatesDirEnvBasicTest struct{}
// Execute executes a test case and returns an error if occurred
func (h *templatesDirEnvBasicTest) Execute(filePath string) error {
tempdir, err := os.MkdirTemp("", "nuclei-templates-dir-env-*")
if err != nil {
return errkit.Wrap(err, "failed to create temp dir")
}
defer func() {
_ = os.RemoveAll(tempdir)
}()
// Copy template to temp directory
if err := copyTemplateToDir(filePath, tempdir); err != nil {
return err
}
// Set NUCLEI_TEMPLATES_DIR and run nuclei
envVars := []string{"NUCLEI_TEMPLATES_DIR=" + tempdir}
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, envVars, "-t", filePath, "-u", "8x8exch02.8x8.com")
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// templatesDirEnvAbsolutePathTest tests that absolute paths work correctly
type templatesDirEnvAbsolutePathTest struct{}
// Execute executes a test case and returns an error if occurred
func (h *templatesDirEnvAbsolutePathTest) Execute(filePath string) error {
tempdir, err := os.MkdirTemp("", "nuclei-templates-dir-env-abs-*")
if err != nil {
return errkit.Wrap(err, "failed to create temp dir")
}
defer func() {
_ = os.RemoveAll(tempdir)
}()
// Get absolute path
absTempDir, err := filepath.Abs(tempdir)
if err != nil {
return errkit.Wrap(err, "failed to get absolute path")
}
// Copy template to temp directory
if err := copyTemplateToDir(filePath, absTempDir); err != nil {
return err
}
// Set NUCLEI_TEMPLATES_DIR with absolute path and run nuclei
envVars := []string{"NUCLEI_TEMPLATES_DIR=" + absTempDir}
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, envVars, "-t", filePath, "-u", "8x8exch02.8x8.com")
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// templatesDirEnvRelativePathTest tests that relative paths are resolved correctly
type templatesDirEnvRelativePathTest struct{}
// Execute executes a test case and returns an error if occurred
func (h *templatesDirEnvRelativePathTest) Execute(filePath string) error {
// Create temp directory in current working directory
tempdir, err := os.MkdirTemp(".", "nuclei-templates-dir-env-rel-*")
if err != nil {
return errkit.Wrap(err, "failed to create temp dir")
}
defer func() {
_ = os.RemoveAll(tempdir)
}()
// Get relative path (just the directory name)
relPath := filepath.Base(tempdir)
// Copy template to temp directory
if err := copyTemplateToDir(filePath, tempdir); err != nil {
return err
}
// Set NUCLEI_TEMPLATES_DIR with relative path and run nuclei
// Note: The implementation should convert relative paths to absolute
envVars := []string{"NUCLEI_TEMPLATES_DIR=" + relPath}
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, envVars, "-t", filePath, "-u", "8x8exch02.8x8.com")
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// templatesDirEnvPrecedenceTest tests that -ud flag takes precedence over NUCLEI_TEMPLATES_DIR
type templatesDirEnvPrecedenceTest struct{}
// Execute executes a test case and returns an error if occurred
func (h *templatesDirEnvPrecedenceTest) Execute(filePath string) error {
// Create two temp directories
envTempDir, err := os.MkdirTemp("", "nuclei-templates-dir-env-*")
if err != nil {
return errkit.Wrap(err, "failed to create env temp dir")
}
defer func() {
_ = os.RemoveAll(envTempDir)
}()
flagTempDir, err := os.MkdirTemp("", "nuclei-templates-dir-flag-*")
if err != nil {
return errkit.Wrap(err, "failed to create flag temp dir")
}
defer func() {
_ = os.RemoveAll(flagTempDir)
}()
// Copy template to flag temp directory (this should be used due to precedence)
if err := copyTemplateToDir(filePath, flagTempDir); err != nil {
return err
}
// Set NUCLEI_TEMPLATES_DIR to envTempDir (should be ignored due to -ud flag)
envVars := []string{"NUCLEI_TEMPLATES_DIR=" + envTempDir}
// Use -ud flag which should take precedence
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, envVars, "-t", filePath, "-u", "8x8exch02.8x8.com", "-ud", flagTempDir)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// templatesDirEnvCustomTemplatesTest tests that custom template subdirectories are correctly set
type templatesDirEnvCustomTemplatesTest struct{}
// Execute executes a test case and returns an error if occurred
func (h *templatesDirEnvCustomTemplatesTest) Execute(filePath string) error {
tempdir, err := os.MkdirTemp("", "nuclei-templates-dir-custom-*")
if err != nil {
return errkit.Wrap(err, "failed to create temp dir")
}
defer func() {
_ = os.RemoveAll(tempdir)
}()
// Create custom template subdirectories structure
customDirs := []string{"github", "s3", "gitlab", "azure"}
for _, dir := range customDirs {
customDirPath := filepath.Join(tempdir, dir)
if err := os.MkdirAll(customDirPath, 0755); err != nil {
return errkit.Wrap(err, "failed to create custom template directory")
}
}
// Copy template to temp directory
if err := copyTemplateToDir(filePath, tempdir); err != nil {
return err
}
// Set NUCLEI_TEMPLATES_DIR and run nuclei
envVars := []string{"NUCLEI_TEMPLATES_DIR=" + tempdir}
results, err := testutils.RunNucleiBareArgsAndGetResults(debug, envVars, "-t", filePath, "-u", "8x8exch02.8x8.com")
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
+12 -12
View File
@@ -62,7 +62,7 @@ type workflowBasic struct{}
func (h *workflowBasic) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -81,7 +81,7 @@ type workflowConditionMatched struct{}
func (h *workflowConditionMatched) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -100,7 +100,7 @@ type workflowConditionUnmatch struct{}
func (h *workflowConditionUnmatch) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -119,7 +119,7 @@ type workflowMatcherName struct{}
func (h *workflowMatcherName) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -138,7 +138,7 @@ type workflowComplexConditions struct{}
func (h *workflowComplexConditions) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -162,11 +162,11 @@ type workflowHttpKeyValueShare struct{}
func (h *workflowHttpKeyValueShare) Execute(filePath string) error {
router := httprouter.New()
router.GET("/path1", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "href=\"test-value\"")
fmt.Fprintf(w, "href=\"test-value\"")
})
router.GET("/path2", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
body, _ := io.ReadAll(r.Body)
_, _ = fmt.Fprintf(w, "%s", body)
fmt.Fprintf(w, "%s", body)
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -214,11 +214,11 @@ func (h *workflowMultiProtocolKeyValueShare) Execute(filePath string) error {
router := httprouter.New()
// the response of path1 contains a domain that will be extracted and shared with the second template
router.GET("/path1", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "href=\"blog.projectdiscovery.io\"")
fmt.Fprintf(w, "href=\"blog.projectdiscovery.io\"")
})
// path2 responds with the value of the "extracted" query parameter, e.g.: /path2?extracted=blog.projectdiscovery.io => blog.projectdiscovery.io
router.GET("/path2", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "%s", r.URL.Query().Get("extracted"))
fmt.Fprintf(w, "%s", r.URL.Query().Get("extracted"))
})
ts := httptest.NewServer(router)
defer ts.Close()
@@ -238,15 +238,15 @@ func (h *workflowMultiMatchKeyValueShare) Execute(filePath string) error {
var receivedData []string
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
fmt.Fprintf(w, "This is test matcher text")
})
router.GET("/path1", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "href=\"test-value-%s\"", r.URL.Query().Get("v"))
fmt.Fprintf(w, "href=\"test-value-%s\"", r.URL.Query().Get("v"))
})
router.GET("/path2", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
body, _ := io.ReadAll(r.Body)
receivedData = append(receivedData, string(body))
_, _ = fmt.Fprintf(w, "test-value")
fmt.Fprintf(w, "test-value")
})
ts := httptest.NewServer(router)
defer ts.Close()
+3 -16
View File
@@ -55,12 +55,6 @@
# # these severity labels or tags (does not affect exporters. set those globally)
# deny-list:
# severity: low
# # duplicate-issue-check (optional) flag to enable duplicate tracking issue check
# duplicate-issue-check: false
# # duplicate-issue-page-size (optional) controls how many issues to fetch per page when searching for duplicates
# duplicate-issue-page-size: 100
# # duplicate-issue-max-pages (optional) limits how many pages to fetch when searching for duplicates (0 = no limit)
# duplicate-issue-max-pages: 0
#
# Gitea contains configuration options for a gitea issue tracker
#gitea:
@@ -87,10 +81,6 @@
# severity: low
# # duplicate-issue-check (optional) flag to enable duplicate tracking issue check
# duplicate-issue-check: false
# # duplicate-issue-page-size (optional) controls how many issues to fetch per page when searching for duplicates
# duplicate-issue-page-size: 100
# # duplicate-issue-max-pages (optional) limits how many pages to fetch when searching for duplicates (0 = no limit)
# duplicate-issue-max-pages: 0
#
# Jira contains configuration options for Jira issue tracker
#jira:
@@ -100,9 +90,6 @@
# update-existing: false
# # URL is the jira application url
# url: https://localhost/jira
# # site-url is the browsable URL for the Jira instance (optional)
# # If not provided, issue.Self will be used. Useful for OAuth where issue.Self contains api.atlassian.com
# site-url: https://your-company.atlassian.net
# # account-id is the account-id of the Jira user or username in case of on-prem Jira
# account-id: test-account-id
# # email is the email of the user for Jira instance
@@ -131,10 +118,10 @@
# status-not: Closed
# # Customfield supports name, id and freeform. name and id are to be used when the custom field is a dropdown.
# # freeform can be used if the custom field is just a text entry
# # Variables can be used to pull various pieces of data from the finding itself.
# # Variables can be used to pull various pieces of data from the finding itself.
# # Supported variables: $CVSSMetrics, $CVEID, $CWEID, $Host, $Severity, $CVSSScore, $Name
# custom-fields:
# customfield_00001:
# customfield_00001:
# name: "Nuclei"
# customfield_00002:
# freeform: $CVSSMetrics
@@ -185,4 +172,4 @@
# omit-raw: false
# # determines the number of results to be kept in memory before writing it to the database or 0 to
# # persist all in memory and write all results at the end (default)
# batch-size: 0
# batch-size: 0
+83 -127
View File
@@ -13,16 +13,14 @@ import (
"strings"
"time"
"github.com/projectdiscovery/gologger"
_pdcp "github.com/projectdiscovery/nuclei/v3/internal/pdcp"
"github.com/projectdiscovery/utils/auth/pdcp"
"github.com/projectdiscovery/utils/env"
_ "github.com/projectdiscovery/utils/pprof"
stringsutil "github.com/projectdiscovery/utils/strings"
"github.com/rs/xid"
"gopkg.in/yaml.v2"
"github.com/projectdiscovery/goflags"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/interactsh/pkg/client"
"github.com/projectdiscovery/nuclei/v3/internal/runner"
@@ -40,7 +38,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/types/scanstrategy"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/monitor"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
fileutil "github.com/projectdiscovery/utils/file"
unitutils "github.com/projectdiscovery/utils/unit"
updateutils "github.com/projectdiscovery/utils/update"
@@ -54,18 +52,16 @@ var (
)
func main() {
options.Logger = gologger.DefaultLogger
// enables CLI specific configs mostly interactive behavior
config.CurrentAppMode = config.AppModeCLI
if err := runner.ConfigureOptions(); err != nil {
options.Logger.Fatal().Msgf("Could not initialize options: %s\n", err)
gologger.Fatal().Msgf("Could not initialize options: %s\n", err)
}
_ = readConfig()
if options.ListDslSignatures {
options.Logger.Info().Msgf("The available custom DSL functions are:")
gologger.Info().Msgf("The available custom DSL functions are:")
fmt.Println(dsl.GetPrintableDslFunctionSignatures(options.NoColor))
return
}
@@ -76,7 +72,7 @@ func main() {
templates.UseOptionsForSigner(options)
tsigner, err := signer.NewTemplateSigner(nil, nil) // will read from env , config or generate new keys
if err != nil {
options.Logger.Fatal().Msgf("couldn't initialize signer crypto engine: %s\n", err)
gologger.Fatal().Msgf("couldn't initialize signer crypto engine: %s\n", err)
}
successCounter := 0
@@ -92,7 +88,7 @@ func main() {
if err != templates.ErrNotATemplate {
// skip warnings and errors as given items are not templates
errorCounter++
options.Logger.Error().Msgf("could not sign '%s': %s\n", iterItem, err)
gologger.Error().Msgf("could not sign '%s': %s\n", iterItem, err)
}
} else {
successCounter++
@@ -101,10 +97,10 @@ func main() {
return nil
})
if err != nil {
options.Logger.Error().Msgf("%s\n", err)
gologger.Error().Msgf("%s\n", err)
}
}
options.Logger.Info().Msgf("All templates signatures were elaborated success=%d failed=%d\n", successCounter, errorCounter)
gologger.Info().Msgf("All templates signatures were elaborated success=%d failed=%d\n", successCounter, errorCounter)
return
}
@@ -115,7 +111,7 @@ func main() {
createProfileFile := func(ext, profileType string) *os.File {
f, err := os.Create(memProfile + ext)
if err != nil {
options.Logger.Fatal().Msgf("profile: could not create %s profile %q file: %v", profileType, f.Name(), err)
gologger.Fatal().Msgf("profile: could not create %s profile %q file: %v", profileType, f.Name(), err)
}
return f
}
@@ -129,47 +125,45 @@ func main() {
// Start tracing
if err := trace.Start(traceFile); err != nil {
options.Logger.Fatal().Msgf("profile: could not start trace: %v", err)
gologger.Fatal().Msgf("profile: could not start trace: %v", err)
}
// Start CPU profiling
if err := pprof.StartCPUProfile(cpuProfileFile); err != nil {
options.Logger.Fatal().Msgf("profile: could not start CPU profile: %v", err)
gologger.Fatal().Msgf("profile: could not start CPU profile: %v", err)
}
defer func() {
// Start heap memory snapshot
if err := pprof.WriteHeapProfile(memProfileFile); err != nil {
options.Logger.Fatal().Msgf("profile: could not write memory profile: %v", err)
gologger.Fatal().Msgf("profile: could not write memory profile: %v", err)
}
pprof.StopCPUProfile()
_ = memProfileFile.Close()
_ = traceFile.Close()
memProfileFile.Close()
traceFile.Close()
trace.Stop()
runtime.MemProfileRate = oldMemProfileRate
options.Logger.Info().Msgf("CPU profile saved at %q", cpuProfileFile.Name())
options.Logger.Info().Msgf("Memory usage snapshot saved at %q", memProfileFile.Name())
options.Logger.Info().Msgf("Traced at %q", traceFile.Name())
gologger.Info().Msgf("CPU profile saved at %q", cpuProfileFile.Name())
gologger.Info().Msgf("Memory usage snapshot saved at %q", memProfileFile.Name())
gologger.Info().Msgf("Traced at %q", traceFile.Name())
}()
}
options.ExecutionId = xid.New().String()
runner.ParseOptions(options)
if options.ScanUploadFile != "" {
if err := runner.UploadResultsToCloud(options); err != nil {
options.Logger.Fatal().Msgf("could not upload scan results to cloud dashboard: %s\n", err)
gologger.Fatal().Msgf("could not upload scan results to cloud dashboard: %s\n", err)
}
return
}
nucleiRunner, err := runner.New(options)
if err != nil {
options.Logger.Fatal().Msgf("Could not create runner: %s\n", err)
gologger.Fatal().Msgf("Could not create runner: %s\n", err)
}
if nucleiRunner == nil {
return
@@ -182,59 +176,58 @@ func main() {
stackMonitor.RegisterCallback(func(dumpID string) error {
resumeFileName := fmt.Sprintf("crash-resume-file-%s.dump", dumpID)
if options.EnableCloudUpload {
options.Logger.Info().Msgf("Uploading scan results to cloud...")
gologger.Info().Msgf("Uploading scan results to cloud...")
}
nucleiRunner.Close()
options.Logger.Info().Msgf("Creating resume file: %s\n", resumeFileName)
gologger.Info().Msgf("Creating resume file: %s\n", resumeFileName)
err := nucleiRunner.SaveResumeConfig(resumeFileName)
if err != nil {
return errkit.Wrap(err, "couldn't create crash resume file")
return errorutil.NewWithErr(err).Msgf("couldn't create crash resume file")
}
return nil
})
}
// Setup filename for graceful exits
// Setup graceful exits
resumeFileName := types.DefaultResumeFilePath()
if options.Resume == "" {
resumeFileName = options.Resume
}
c := make(chan os.Signal, 1)
defer close(c)
signal.Notify(c, os.Interrupt)
go func() {
<-c
options.Logger.Info().Msgf("CTRL+C pressed: Exiting\n")
if options.DASTServer {
for range c {
gologger.Info().Msgf("CTRL+C pressed: Exiting\n")
if options.DASTServer {
nucleiRunner.Close()
os.Exit(1)
}
gologger.Info().Msgf("Attempting graceful shutdown...")
if options.EnableCloudUpload {
gologger.Info().Msgf("Uploading scan results to cloud...")
}
nucleiRunner.Close()
if options.ShouldSaveResume() {
gologger.Info().Msgf("Creating resume file: %s\n", resumeFileName)
err := nucleiRunner.SaveResumeConfig(resumeFileName)
if err != nil {
gologger.Error().Msgf("Couldn't create resume file: %s\n", err)
}
}
os.Exit(1)
}
options.Logger.Info().Msgf("Attempting graceful shutdown...")
if options.EnableCloudUpload {
options.Logger.Info().Msgf("Uploading scan results to cloud...")
}
nucleiRunner.Close()
if options.ShouldSaveResume() {
options.Logger.Info().Msgf("Creating resume file: %s\n", resumeFileName)
err := nucleiRunner.SaveResumeConfig(resumeFileName)
if err != nil {
options.Logger.Error().Msgf("Couldn't create resume file: %s\n", err)
}
}
os.Exit(1)
}()
if err := nucleiRunner.RunEnumeration(); err != nil {
if options.Validate {
options.Logger.Fatal().Msgf("Could not validate templates: %s\n", err)
gologger.Fatal().Msgf("Could not validate templates: %s\n", err)
} else {
options.Logger.Fatal().Msgf("Could not run nuclei: %s\n", err)
gologger.Fatal().Msgf("Could not run nuclei: %s\n", err)
}
}
nucleiRunner.Close()
// on successful execution remove the resume file in case it exists
if fileutil.FileExists(resumeFileName) {
_ = os.Remove(resumeFileName)
os.Remove(resumeFileName)
}
}
@@ -258,7 +251,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringSliceVarP(&options.Targets, "target", "u", nil, "target URLs/hosts to scan", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringVarP(&options.TargetsFilePath, "list", "l", "", "path to file containing a list of target URLs/hosts to scan (one per line)"),
flagSet.StringSliceVarP(&options.ExcludeTargets, "exclude-hosts", "eh", nil, "hosts to exclude to scan from the input list (ip, cidr, hostname)", goflags.FileCommaSeparatedStringSliceOptions),
flagSet.StringVar(&options.Resume, "resume", "", "resume scan from and save to specified file (clustering will be disabled)"),
flagSet.StringVar(&options.Resume, "resume", "", "resume scan using resume.cfg (clustering will be disabled)"),
flagSet.BoolVarP(&options.ScanAllIPs, "scan-all-ips", "sa", false, "scan all the IP's associated with dns record"),
flagSet.StringSliceVarP(&options.IPVersion, "ip-version", "iv", nil, "IP version to scan of hostname (4,6) - (default 4)", goflags.CommaSeparatedStringSliceOptions),
)
@@ -267,8 +260,6 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringVarP(&options.InputFileMode, "input-mode", "im", "list", fmt.Sprintf("mode of input file (%v)", provider.SupportedInputFormats())),
flagSet.BoolVarP(&options.FormatUseRequiredOnly, "required-only", "ro", false, "use only required fields in input format when generating requests"),
flagSet.BoolVarP(&options.SkipFormatValidation, "skip-format-validation", "sfv", false, "skip format validation (like missing vars) when parsing input file"),
flagSet.BoolVarP(&options.VarsTextTemplating, "vars-text-templating", "vtt", false, "enable text templating for vars in input file (only for yaml input mode)"),
flagSet.StringSliceVarP(&options.VarsFilePaths, "var-file-paths", "vfp", nil, "list of yaml file contained vars to inject into yaml input", goflags.CommaSeparatedStringSliceOptions),
)
flagSet.CreateGroup("templates", "Templates",
@@ -283,7 +274,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVar(&options.Validate, "validate", false, "validate the passed templates to nuclei"),
flagSet.BoolVarP(&options.NoStrictSyntax, "no-strict-syntax", "nss", false, "disable strict syntax check on templates"),
flagSet.BoolVarP(&options.TemplateDisplay, "template-display", "td", false, "displays the templates content"),
flagSet.BoolVar(&options.TemplateList, "tl", false, "list all templates matching current filters"),
flagSet.BoolVar(&options.TemplateList, "tl", false, "list all available templates"),
flagSet.BoolVar(&options.TagList, "tgl", false, "list all available tags"),
flagSet.StringSliceVarConfigOnly(&options.RemoteTemplateDomainList, "remote-template-domain", []string{"cloud.projectdiscovery.io"}, "allowed domain list to load remote templates from"),
flagSet.BoolVar(&options.SignTemplates, "sign", false, "signs the templates with the private key defined in NUCLEI_SIGNATURE_PRIVATE_KEY env variable"),
@@ -414,7 +405,6 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.IntVarP(&options.JsConcurrency, "js-concurrency", "jsc", 120, "maximum number of javascript runtimes to be executed in parallel"),
flagSet.IntVarP(&options.PayloadConcurrency, "payload-concurrency", "pc", 25, "max payload concurrency for each template"),
flagSet.IntVarP(&options.ProbeConcurrency, "probe-concurrency", "prc", 50, "http probe concurrency with httpx"),
flagSet.IntVarP(&options.TemplateLoadingConcurrency, "template-loading-concurrency", "tlc", types.DefaultTemplateLoadingConcurrency, "maximum number of concurrent template loading operations"),
)
flagSet.CreateGroup("optimization", "Optimizations",
flagSet.IntVar(&options.Timeout, "timeout", 10, "time to wait in seconds before timeout"),
@@ -552,11 +542,11 @@ Additional documentation is available at: https://docs.nuclei.sh/getting-started
h := &pdcp.PDCPCredHandler{}
_, err := h.GetCreds()
if err != nil {
options.Logger.Fatal().Msg("To utilize the `-ai` flag, please configure your API key with the `-auth` flag or set the `PDCP_API_KEY` environment variable")
gologger.Fatal().Msg("To utilize the `-ai` flag, please configure your API key with the `-auth` flag or set the `PDCP_API_KEY` environment variable")
}
}
options.Logger.SetTimestamp(options.Timestamp, levels.LevelDebug)
gologger.DefaultLogger.SetTimestamp(options.Timestamp, levels.LevelDebug)
if options.VerboseVerbose {
// hide release notes if silent mode is enabled
@@ -578,58 +568,17 @@ Additional documentation is available at: https://docs.nuclei.sh/getting-started
config.DefaultConfig.SetConfigDir(customConfigDir)
readFlagsConfig(flagSet)
}
if cfgFile != "" {
if !fileutil.FileExists(cfgFile) {
options.Logger.Fatal().Msgf("given config file '%s' does not exist", cfgFile)
gologger.Fatal().Msgf("given config file '%s' does not exist", cfgFile)
}
// merge config file with flags
if err := flagSet.MergeConfigFile(cfgFile); err != nil {
options.Logger.Fatal().Msgf("Could not read config: %s\n", err)
}
if !options.Vars.IsEmpty() {
// Maybe we should add vars to the config file as well even if they are set via flags?
file, err := os.Open(cfgFile)
if err != nil {
gologger.Fatal().Msgf("Could not open config file: %s\n", err)
}
defer func() {
_ = file.Close()
}()
data := make(map[string]interface{})
err = yaml.NewDecoder(file).Decode(&data)
if err != nil {
gologger.Fatal().Msgf("Could not decode config file: %s\n", err)
}
variables := data["var"]
if variables != nil {
if varSlice, ok := variables.([]interface{}); ok {
for _, value := range varSlice {
if strVal, ok := value.(string); ok {
err = options.Vars.Set(strVal)
if err != nil {
gologger.Warning().Msgf("Could not set variable from config file: %s\n", err)
}
} else {
gologger.Warning().Msgf("Skipping non-string variable in config: %#v", value)
}
}
} else {
gologger.Warning().Msgf("No 'var' section found in config file: %s", cfgFile)
}
}
gologger.Fatal().Msgf("Could not read config: %s\n", err)
}
}
templatesDir := options.NewTemplatesDirectory
if templatesDir == "" {
templatesDir = os.Getenv(config.NucleiTemplatesDirEnv)
}
if templatesDir != "" {
config.DefaultConfig.SetTemplatesDir(templatesDir)
if options.NewTemplatesDirectory != "" {
config.DefaultConfig.SetTemplatesDir(options.NewTemplatesDirectory)
}
defaultProfilesPath := filepath.Join(config.DefaultConfig.GetTemplateDir(), "profiles")
@@ -638,7 +587,7 @@ Additional documentation is available at: https://docs.nuclei.sh/getting-started
if tp := findProfilePathById(templateProfile, defaultProfilesPath); tp != "" {
templateProfile = tp
} else {
options.Logger.Fatal().Msgf("'%s' is not a profile-id or profile path", templateProfile)
gologger.Fatal().Msgf("'%s' is not a profile-id or profile path", templateProfile)
}
}
if !filepath.IsAbs(templateProfile) {
@@ -653,17 +602,17 @@ Additional documentation is available at: https://docs.nuclei.sh/getting-started
}
}
if !fileutil.FileExists(templateProfile) {
options.Logger.Fatal().Msgf("given template profile file '%s' does not exist", templateProfile)
gologger.Fatal().Msgf("given template profile file '%s' does not exist", templateProfile)
}
if err := flagSet.MergeConfigFile(templateProfile); err != nil {
options.Logger.Fatal().Msgf("Could not read template profile: %s\n", err)
gologger.Fatal().Msgf("Could not read template profile: %s\n", err)
}
}
if len(options.SecretsFile) > 0 {
for _, secretFile := range options.SecretsFile {
if !fileutil.FileExists(secretFile) {
options.Logger.Fatal().Msgf("given secrets file '%s' does not exist", secretFile)
gologger.Fatal().Msgf("given secrets file '%s' does not exist", options.SecretsFile)
}
}
}
@@ -689,25 +638,25 @@ func readFlagsConfig(flagset *goflags.FlagSet) {
if err != nil {
// something went wrong either dir is not readable or something else went wrong upstream in `goflags`
// warn and exit in this case
options.Logger.Warning().Msgf("Could not read config file: %s\n", err)
gologger.Warning().Msgf("Could not read config file: %s\n", err)
return
}
cfgFile := config.DefaultConfig.GetFlagsConfigFilePath()
if !fileutil.FileExists(cfgFile) {
if !fileutil.FileExists(defaultCfgFile) {
// if default config does not exist, warn and exit
options.Logger.Warning().Msgf("missing default config file : %s", defaultCfgFile)
gologger.Warning().Msgf("missing default config file : %s", defaultCfgFile)
return
}
// if does not exist copy it from the default config
if err = fileutil.CopyFile(defaultCfgFile, cfgFile); err != nil {
options.Logger.Warning().Msgf("Could not copy config file: %s\n", err)
gologger.Warning().Msgf("Could not copy config file: %s\n", err)
}
return
}
// if config file exists, merge it with the default config
if err = flagset.MergeConfigFile(cfgFile); err != nil {
options.Logger.Warning().Msgf("failed to merge configfile with flags got: %s\n", err)
gologger.Warning().Msgf("failed to merge configfile with flags got: %s\n", err)
}
}
@@ -718,29 +667,29 @@ func disableUpdatesCallback() {
// printVersion prints the nuclei version and exits.
func printVersion() {
options.Logger.Info().Msgf("Nuclei Engine Version: %s", config.Version)
options.Logger.Info().Msgf("Nuclei Config Directory: %s", config.DefaultConfig.GetConfigDir())
options.Logger.Info().Msgf("Nuclei Cache Directory: %s", config.DefaultConfig.GetCacheDir()) // cache dir contains resume files
options.Logger.Info().Msgf("PDCP Directory: %s", pdcp.PDCPDir)
gologger.Info().Msgf("Nuclei Engine Version: %s", config.Version)
gologger.Info().Msgf("Nuclei Config Directory: %s", config.DefaultConfig.GetConfigDir())
gologger.Info().Msgf("Nuclei Cache Directory: %s", config.DefaultConfig.GetCacheDir()) // cache dir contains resume files
gologger.Info().Msgf("PDCP Directory: %s", pdcp.PDCPDir)
os.Exit(0)
}
// printTemplateVersion prints the nuclei template version and exits.
func printTemplateVersion() {
cfg := config.DefaultConfig
options.Logger.Info().Msgf("Public nuclei-templates version: %s (%s)\n", cfg.TemplateVersion, cfg.TemplatesDirectory)
gologger.Info().Msgf("Public nuclei-templates version: %s (%s)\n", cfg.TemplateVersion, cfg.TemplatesDirectory)
if fileutil.FolderExists(cfg.CustomS3TemplatesDirectory) {
options.Logger.Info().Msgf("Custom S3 templates location: %s\n", cfg.CustomS3TemplatesDirectory)
gologger.Info().Msgf("Custom S3 templates location: %s\n", cfg.CustomS3TemplatesDirectory)
}
if fileutil.FolderExists(cfg.CustomGitHubTemplatesDirectory) {
options.Logger.Info().Msgf("Custom GitHub templates location: %s ", cfg.CustomGitHubTemplatesDirectory)
gologger.Info().Msgf("Custom GitHub templates location: %s ", cfg.CustomGitHubTemplatesDirectory)
}
if fileutil.FolderExists(cfg.CustomGitLabTemplatesDirectory) {
options.Logger.Info().Msgf("Custom GitLab templates location: %s ", cfg.CustomGitLabTemplatesDirectory)
gologger.Info().Msgf("Custom GitLab templates location: %s ", cfg.CustomGitLabTemplatesDirectory)
}
if fileutil.FolderExists(cfg.CustomAzureTemplatesDirectory) {
options.Logger.Info().Msgf("Custom Azure templates location: %s ", cfg.CustomAzureTemplatesDirectory)
gologger.Info().Msgf("Custom Azure templates location: %s ", cfg.CustomAzureTemplatesDirectory)
}
os.Exit(0)
}
@@ -756,13 +705,13 @@ Following files will be deleted:
Note: Make sure you have backup of your custom nuclei-templates before proceeding
`, config.DefaultConfig.GetConfigDir(), config.DefaultConfig.TemplatesDirectory)
options.Logger.Print().Msg(warning)
gologger.Print().Msg(warning)
reader := bufio.NewReader(os.Stdin)
for {
fmt.Print("Are you sure you want to continue? [y/n]: ")
resp, err := reader.ReadString('\n')
if err != nil {
options.Logger.Fatal().Msgf("could not read response: %s", err)
gologger.Fatal().Msgf("could not read response: %s", err)
}
resp = strings.TrimSpace(resp)
if stringsutil.EqualFoldAny(resp, "y", "yes") {
@@ -775,13 +724,13 @@ Note: Make sure you have backup of your custom nuclei-templates before proceedin
}
err := os.RemoveAll(config.DefaultConfig.GetConfigDir())
if err != nil {
options.Logger.Fatal().Msgf("could not delete config dir: %s", err)
gologger.Fatal().Msgf("could not delete config dir: %s", err)
}
err = os.RemoveAll(config.DefaultConfig.TemplatesDirectory)
if err != nil {
options.Logger.Fatal().Msgf("could not delete templates dir: %s", err)
gologger.Fatal().Msgf("could not delete templates dir: %s", err)
}
options.Logger.Info().Msgf("Successfully deleted all nuclei configurations files and nuclei-templates")
gologger.Info().Msgf("Successfully deleted all nuclei configurations files and nuclei-templates")
os.Exit(0)
}
@@ -801,7 +750,14 @@ func findProfilePathById(profileId, templatesDir string) string {
return nil
})
if err != nil && err.Error() != "FOUND" {
options.Logger.Error().Msgf("%s\n", err)
gologger.Error().Msgf("%s\n", err)
}
return profilePath
}
func init() {
// print stacktrace of errors in debug mode
if strings.EqualFold(os.Getenv("DEBUG"), "true") {
errorutil.ShowStackTrace = true
}
}
-196
View File
@@ -1,196 +0,0 @@
package main_test
import (
"fmt"
"net/http"
"net/http/httptest"
"os"
"runtime"
"runtime/pprof"
"strings"
"testing"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/internal/runner"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
var (
projectPath string
targetURL string
)
func TestMain(m *testing.M) {
// Set up
gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent)
_ = os.Setenv("DISABLE_STDOUT", "true")
var err error
projectPath, err = os.MkdirTemp("", "nuclei-benchmark-")
if err != nil {
panic(err)
}
dummyServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
targetURL = dummyServer.URL
// Execute tests
exitCode := m.Run()
// Tear down
dummyServer.Close()
_ = os.RemoveAll(projectPath)
_ = os.Unsetenv("DISABLE_STDOUT")
os.Exit(exitCode)
}
// getUniqFilename generates a unique filename by appending .N if file exists
// Similar to wget's behavior: file.cpu.prof, file.cpu.1.prof, file.cpu.2.prof, etc.
func getUniqFilename(basePath string) string {
if _, err := os.Stat(basePath); os.IsNotExist(err) {
return basePath
}
lastDot := strings.LastIndex(basePath, ".")
var name, ext string
if lastDot != -1 {
name = basePath[:lastDot]
ext = basePath[lastDot:]
} else {
name = basePath
ext = ""
}
for i := 1; ; i++ {
newPath := fmt.Sprintf("%s.%d%s", name, i, ext)
if _, err := os.Stat(newPath); os.IsNotExist(err) {
return newPath
}
}
}
func getDefaultOptions() *types.Options {
return &types.Options{
RemoteTemplateDomainList: []string{"cloud.projectdiscovery.io"},
ProjectPath: projectPath,
StatsInterval: 5,
MetricsPort: 9092,
MaxHostError: 30,
NoHostErrors: true,
BulkSize: 25,
TemplateThreads: 25,
HeadlessBulkSize: 10,
HeadlessTemplateThreads: 10,
Timeout: 10,
Retries: 1,
RateLimit: 150,
RateLimitDuration: time.Duration(time.Second),
RateLimitMinute: 0,
PageTimeout: 20,
InteractionsCacheSize: 5000,
InteractionsPollDuration: 5,
InteractionsEviction: 60,
InteractionsCoolDownPeriod: 5,
MaxRedirects: 10,
Silent: true,
VarDumpLimit: 255,
JSONRequests: true,
StoreResponseDir: "output",
InputFileMode: "list",
ResponseReadSize: 0,
ResponseSaveSize: 1048576,
InputReadTimeout: time.Duration(3 * time.Minute),
UncoverField: "ip:port",
UncoverLimit: 100,
UncoverRateLimit: 60,
ScanStrategy: "auto",
FuzzAggressionLevel: "low",
FuzzParamFrequency: 10,
TeamID: "none",
JsConcurrency: 120,
PayloadConcurrency: 25,
ProbeConcurrency: 50,
LoadHelperFileFunction: types.DefaultOptions().LoadHelperFileFunction,
// DialerKeepAlive: time.Duration(0),
// DASTServerAddress: "localhost:9055",
ExecutionId: "test",
Logger: gologger.DefaultLogger,
}
}
func runEnumBenchmark(b *testing.B, options *types.Options) {
runner.ParseOptions(options)
nucleiRunner, err := runner.New(options)
if err != nil {
b.Fatalf("failed to create runner: %s", err)
}
defer nucleiRunner.Close()
benchNameSlug := strings.ReplaceAll(b.Name(), "/", "-")
// Start CPU profiling
cpuProfileBase := fmt.Sprintf("%s.cpu.prof", benchNameSlug)
cpuProfilePath := getUniqFilename(cpuProfileBase)
cpuProfile, err := os.Create(cpuProfilePath)
if err != nil {
b.Fatalf("failed to create CPU profile: %s", err)
}
defer func() { _ = cpuProfile.Close() }()
if err := pprof.StartCPUProfile(cpuProfile); err != nil {
b.Fatalf("failed to start CPU profile: %s", err)
}
defer pprof.StopCPUProfile()
b.ReportAllocs()
for b.Loop() {
if err := nucleiRunner.RunEnumeration(); err != nil {
b.Fatalf("%s failed: %s", b.Name(), err)
}
}
b.StopTimer()
// Write heap profile
heapProfileBase := fmt.Sprintf("%s.heap.prof", benchNameSlug)
heapProfilePath := getUniqFilename(heapProfileBase)
heapProfile, err := os.Create(heapProfilePath)
if err != nil {
b.Fatalf("failed to create heap profile: %s", err)
}
defer func() { _ = heapProfile.Close() }()
runtime.GC() // Force GC before heap profile
if err := pprof.WriteHeapProfile(heapProfile); err != nil {
b.Fatalf("failed to write heap profile: %s", err)
}
}
func BenchmarkRunEnumeration(b *testing.B) {
// Default case: run enumeration with default options == all nuclei-templates
b.Run("Default", func(b *testing.B) {
options := getDefaultOptions()
options.Targets = []string{targetURL}
runEnumBenchmark(b, options)
})
// Case: https://github.com/projectdiscovery/nuclei/pull/6258
b.Run("Multiproto", func(b *testing.B) {
options := getDefaultOptions()
options.Targets = []string{targetURL}
options.Templates = []string{"./cmd/nuclei/testdata/benchmark/multiproto/"}
runEnumBenchmark(b, options)
})
}
+87
View File
@@ -0,0 +1,87 @@
package main_test
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/projectdiscovery/goflags"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/internal/runner"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
func BenchmarkRunEnumeration(b *testing.B) {
dummyServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
defer dummyServer.Close()
options := &types.Options{
RemoteTemplateDomainList: goflags.StringSlice{
"cloud.projectdiscovery.io",
},
ProjectPath: "/tmp",
Targets: goflags.StringSlice{dummyServer.URL},
StatsInterval: 5,
MetricsPort: 9092,
MaxHostError: 30,
NoHostErrors: true,
BulkSize: 25,
TemplateThreads: 25,
HeadlessBulkSize: 10,
HeadlessTemplateThreads: 10,
Timeout: 10,
Retries: 1,
RateLimit: 150,
RateLimitDuration: time.Duration(time.Second),
RateLimitMinute: 0,
PageTimeout: 20,
InteractionsCacheSize: 5000,
InteractionsPollDuration: 5,
InteractionsEviction: 60,
InteractionsCoolDownPeriod: 5,
MaxRedirects: 10,
Silent: true,
VarDumpLimit: 255,
JSONRequests: true,
StoreResponseDir: "output",
InputFileMode: "list",
ResponseReadSize: 0,
ResponseSaveSize: 1048576,
InputReadTimeout: time.Duration(3 * time.Minute),
UncoverField: "ip:port",
UncoverLimit: 100,
UncoverRateLimit: 60,
ScanStrategy: "auto",
FuzzAggressionLevel: "low",
FuzzParamFrequency: 10,
TeamID: "none",
JsConcurrency: 120,
PayloadConcurrency: 25,
ProbeConcurrency: 50,
LoadHelperFileFunction: types.DefaultOptions().LoadHelperFileFunction,
// DialerKeepAlive: time.Duration(0),
// DASTServerAddress: "localhost:9055",
}
runner.ParseOptions(options)
// Disable logging to reduce benchmark noise.
gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent)
nucleiRunner, err := runner.New(options)
if err != nil {
b.Fatalf("failed to create runner: %s", err)
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := nucleiRunner.RunEnumeration(); err != nil {
b.Fatalf("RunEnumeration failed: %s", err)
}
}
}
+18
View File
@@ -0,0 +1,18 @@
id: basic-dns-a-example
info:
name: Test DNS A Query Template
author: pdteam
severity: info
dns:
- name: "{{FQDN}}"
type: SRV
class: inet
recursion: true
retries: 3
matchers:
- type: word
part: all
words:
- "SRV"
@@ -1,239 +0,0 @@
id: basic-template-multiproto-mixed
info:
name: Test Template Multiple Protocols (Mixed)
author: pdteam
severity: info
http:
- method: GET
id: first_iter_http
path:
- '{{BaseURL}}/1'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/2'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/3'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/4'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/5'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/6'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/7'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/8'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/9'
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /10 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /11 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /12 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /13 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /14 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET / HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /15 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /16 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /17 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /18 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
@@ -1,292 +0,0 @@
id: basic-template-multiproto-raw
info:
name: Test Template Multiple Protocols RAW
author: pdteam
severity: info
http:
- raw:
- |
GET /1 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /2 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /3 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /4 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /5 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /6 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /7 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /8 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /9 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /10 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /11 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /12 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /13 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /14 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET / HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /15 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /16 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /17 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
- raw:
- |
GET /18 HTTP/1.1
Host: {{Hostname}}
Origin: {{BaseURL}}
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9
matchers:
- type: word
words:
- "Test is test matcher text"
@@ -1,170 +0,0 @@
id: basic-template-multiproto
info:
name: Test Template Multiple Protocols
author: pdteam
severity: info
http:
- method: GET
id: first_iter_http
path:
- '{{BaseURL}}/1'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/2'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/3'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/4'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/5'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/6'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/7'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/8'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/9'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/10'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/11'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/12'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/13'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/14'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/15'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/16'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/17'
matchers:
- type: word
words:
- "Test is test matcher text"
- method: GET
path:
- '{{BaseURL}}/18'
matchers:
- type: word
words:
- "Test is test matcher text"
+27 -31
View File
@@ -23,7 +23,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
"gopkg.in/yaml.v3"
)
@@ -135,9 +135,7 @@ func process(opts options) error {
if err != nil {
return err
}
defer func() {
_ = os.RemoveAll(tempDir)
}()
defer os.RemoveAll(tempDir)
var errFile *os.File
if opts.errorLogFile != "" {
@@ -145,9 +143,7 @@ func process(opts options) error {
if err != nil {
gologger.Fatal().Msgf("could not open error log file: %s\n", err)
}
defer func() {
_ = errFile.Close()
}()
defer errFile.Close()
}
templateCatalog := disk.NewCatalog(filepath.Dir(opts.input))
@@ -230,7 +226,7 @@ func logErrMsg(path string, err error, debug bool, errFile *os.File) string {
msg = fmt.Sprintf("❌ template: %s err: %s\n", path, err)
}
if errFile != nil {
_, _ = fmt.Fprintf(errFile, "❌ template: %s err: %s\n", path, err)
_, _ = errFile.WriteString(fmt.Sprintf("❌ template: %s err: %s\n", path, err))
}
return msg
}
@@ -243,7 +239,7 @@ func enhanceTemplate(data string) (string, bool, error) {
return data, false, err
}
if resp.StatusCode != 200 {
return data, false, errkit.New("unexpected status code: %v", resp.Status)
return data, false, errorutil.New("unexpected status code: %v", resp.Status)
}
var templateResp TemplateResp
if err := json.NewDecoder(resp.Body).Decode(&templateResp); err != nil {
@@ -254,20 +250,20 @@ func enhanceTemplate(data string) (string, bool, error) {
}
if templateResp.ValidateErrorCount > 0 {
if len(templateResp.ValidateError) > 0 {
return data, false, errkit.New(templateResp.ValidateError[0].Message+": at line %v", templateResp.ValidateError[0].Mark.Line, "tag", "validate")
return data, false, errorutil.NewWithTag("validate", templateResp.ValidateError[0].Message+": at line %v", templateResp.ValidateError[0].Mark.Line)
}
return data, false, errkit.New("validation failed", "tag", "validate")
return data, false, errorutil.New("validation failed").WithTag("validate")
}
if templateResp.Error.Name != "" {
return data, false, errkit.New("%s", templateResp.Error.Name)
return data, false, errorutil.New("%s", templateResp.Error.Name)
}
if strings.TrimSpace(templateResp.Enhanced) == "" && !templateResp.Lint {
if templateResp.LintError.Reason != "" {
return data, false, errkit.New(templateResp.LintError.Reason+" : at line %v", templateResp.LintError.Mark.Line, "tag", "lint")
return data, false, errorutil.NewWithTag("lint", templateResp.LintError.Reason+" : at line %v", templateResp.LintError.Mark.Line)
}
return data, false, errkit.New("at line: %v", templateResp.LintError.Mark.Line, "tag", "lint")
return data, false, errorutil.NewWithTag("lint", "at line: %v", templateResp.LintError.Mark.Line)
}
return data, false, errkit.New("template enhance failed")
return data, false, errorutil.New("template enhance failed")
}
// formatTemplate formats template data using templateman format api
@@ -277,7 +273,7 @@ func formatTemplate(data string) (string, bool, error) {
return data, false, err
}
if resp.StatusCode != 200 {
return data, false, errkit.New("unexpected status code: %v", resp.Status)
return data, false, errorutil.New("unexpected status code: %v", resp.Status)
}
var templateResp TemplateResp
if err := json.NewDecoder(resp.Body).Decode(&templateResp); err != nil {
@@ -288,20 +284,20 @@ func formatTemplate(data string) (string, bool, error) {
}
if templateResp.ValidateErrorCount > 0 {
if len(templateResp.ValidateError) > 0 {
return data, false, errkit.New(templateResp.ValidateError[0].Message+": at line %v", templateResp.ValidateError[0].Mark.Line, "tag", "validate")
return data, false, errorutil.NewWithTag("validate", templateResp.ValidateError[0].Message+": at line %v", templateResp.ValidateError[0].Mark.Line)
}
return data, false, errkit.New("validation failed", "tag", "validate")
return data, false, errorutil.New("validation failed").WithTag("validate")
}
if templateResp.Error.Name != "" {
return data, false, errkit.New("%s", templateResp.Error.Name)
return data, false, errorutil.New("%s", templateResp.Error.Name)
}
if strings.TrimSpace(templateResp.Updated) == "" && !templateResp.Lint {
if templateResp.LintError.Reason != "" {
return data, false, errkit.New(templateResp.LintError.Reason+" : at line %v", templateResp.LintError.Mark.Line, "tag", "lint")
return data, false, errorutil.NewWithTag("lint", templateResp.LintError.Reason+" : at line %v", templateResp.LintError.Mark.Line)
}
return data, false, errkit.New("at line: %v", templateResp.LintError.Mark.Line, "tag", "lint")
return data, false, errorutil.NewWithTag("lint", "at line: %v", templateResp.LintError.Mark.Line)
}
return data, false, errkit.New("template format failed")
return data, false, errorutil.New("template format failed")
}
// lintTemplate lints template data using templateman lint api
@@ -311,7 +307,7 @@ func lintTemplate(data string) (bool, error) {
return false, err
}
if resp.StatusCode != 200 {
return false, errkit.New("unexpected status code: %v", resp.Status)
return false, errorutil.New("unexpected status code: %v", resp.Status)
}
var lintResp TemplateLintResp
if err := json.NewDecoder(resp.Body).Decode(&lintResp); err != nil {
@@ -321,9 +317,9 @@ func lintTemplate(data string) (bool, error) {
return true, nil
}
if lintResp.LintError.Reason != "" {
return false, errkit.New(lintResp.LintError.Reason+" : at line %v", lintResp.LintError.Mark.Line, "tag", "lint")
return false, errorutil.NewWithTag("lint", lintResp.LintError.Reason+" : at line %v", lintResp.LintError.Mark.Line)
}
return false, errkit.New("at line: %v", lintResp.LintError.Mark.Line, "tag", "lint")
return false, errorutil.NewWithTag("lint", "at line: %v", lintResp.LintError.Mark.Line)
}
// validateTemplate validates template data using templateman validate api
@@ -333,7 +329,7 @@ func validateTemplate(data string) (bool, error) {
return false, err
}
if resp.StatusCode != 200 {
return false, errkit.New("unexpected status code: %v", resp.Status)
return false, errorutil.New("unexpected status code: %v", resp.Status)
}
var validateResp TemplateResp
if err := json.NewDecoder(resp.Body).Decode(&validateResp); err != nil {
@@ -344,14 +340,14 @@ func validateTemplate(data string) (bool, error) {
}
if validateResp.ValidateErrorCount > 0 {
if len(validateResp.ValidateError) > 0 {
return false, errkit.New(validateResp.ValidateError[0].Message+": at line %v", validateResp.ValidateError[0].Mark.Line, "tag", "validate")
return false, errorutil.NewWithTag("validate", validateResp.ValidateError[0].Message+": at line %v", validateResp.ValidateError[0].Mark.Line)
}
return false, errkit.New("validation failed", "tag", "validate")
return false, errorutil.New("validation failed").WithTag("validate")
}
if validateResp.Error.Name != "" {
return false, errkit.New("%s", validateResp.Error.Name)
return false, errorutil.New("%s", validateResp.Error.Name)
}
return false, errkit.New("template validation failed")
return false, errorutil.New("template validation failed")
}
// parseAndAddMaxRequests parses and adds max requests to templates
@@ -401,7 +397,7 @@ func parseAndAddMaxRequests(catalog catalog.Catalog, path, data string) (string,
// parseTemplate parses a template and returns the template object
func parseTemplate(catalog catalog.Catalog, templatePath string) (*templates.Template, error) {
executorOpts := &protocols.ExecutorOptions{
executorOpts := protocols.ExecutorOptions{
Catalog: catalog,
Options: defaultOpts,
}
+1 -3
View File
@@ -18,9 +18,7 @@ func main() {
defer fuzzplayground.Cleanup()
server := fuzzplayground.GetPlaygroundServer()
defer func() {
_ = server.Close()
}()
defer server.Close()
// Start the server
if err := server.Start(addr); err != nil {
+2 -2
View File
@@ -99,12 +99,12 @@ func main() {
gologger.Info().Msgf("✓ Template signed & verified successfully")
}
func defaultExecutorOpts(templatePath string) *protocols.ExecutorOptions {
func defaultExecutorOpts(templatePath string) protocols.ExecutorOptions {
// use parsed options when initializing signer instead of default options
options := types.DefaultOptions()
templates.UseOptionsForSigner(options)
catalog := disk.NewCatalog(filepath.Dir(templatePath))
executerOpts := &protocols.ExecutorOptions{
executerOpts := protocols.ExecutorOptions{
Catalog: catalog,
Options: options,
TemplatePath: templatePath,
+1 -2
View File
@@ -1,7 +1,6 @@
package main
import (
"context"
"log"
"sync"
"time"
@@ -35,7 +34,7 @@ func main() {
}
func initializeNucleiEngine() (*nuclei.NucleiEngine, error) {
return nuclei.NewNucleiEngineCtx(context.TODO(),
return nuclei.NewNucleiEngine(
nuclei.WithTemplateFilters(nuclei.TemplateFilters{Tags: []string{"oast"}}),
nuclei.EnableStatsWithOpts(nuclei.StatsOptions{MetricServerPort: 6064}),
nuclei.WithGlobalRateLimit(1, time.Second),
+197 -254
View File
@@ -1,426 +1,369 @@
module github.com/projectdiscovery/nuclei/v3
go 1.24.2
go 1.23.0
toolchain go1.24.4
toolchain go1.24.1
require (
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible
github.com/andygrunwald/go-jira v1.16.1
github.com/antchfx/htmlquery v1.3.5
github.com/andygrunwald/go-jira v1.16.0
github.com/antchfx/htmlquery v1.3.0
github.com/bluele/gcache v0.0.2
github.com/go-playground/validator/v10 v10.26.0
github.com/go-playground/validator/v10 v10.14.1
github.com/go-rod/rod v0.116.2
github.com/gobwas/ws v1.4.0
github.com/gobwas/ws v1.2.1
github.com/google/go-github v17.0.0+incompatible
github.com/invopop/jsonschema v0.13.0
github.com/itchyny/gojq v0.12.17
github.com/invopop/jsonschema v0.12.0
github.com/itchyny/gojq v0.12.13
github.com/json-iterator/go v1.1.12
github.com/julienschmidt/httprouter v1.3.0
github.com/logrusorgru/aurora v2.0.3+incompatible
github.com/miekg/dns v1.1.68
github.com/olekukonko/tablewriter v1.0.8
github.com/miekg/dns v1.1.62
github.com/olekukonko/tablewriter v0.0.5
github.com/pkg/errors v0.9.1
github.com/projectdiscovery/clistats v0.1.1
github.com/projectdiscovery/fastdialer v0.5.1
github.com/projectdiscovery/hmap v0.0.99
github.com/projectdiscovery/fastdialer v0.4.0
github.com/projectdiscovery/hmap v0.0.88
github.com/projectdiscovery/interactsh v1.2.4
github.com/projectdiscovery/rawhttp v0.1.90
github.com/projectdiscovery/retryabledns v1.0.112
github.com/projectdiscovery/retryablehttp-go v1.3.1
github.com/projectdiscovery/retryabledns v1.0.99
github.com/projectdiscovery/retryablehttp-go v1.0.111
github.com/projectdiscovery/yamldoc-go v1.0.6
github.com/remeh/sizedwaitgroup v1.0.0
github.com/rs/xid v1.6.0
github.com/segmentio/ksuid v1.0.4
github.com/shirou/gopsutil/v3 v3.24.5 // indirect
github.com/shirou/gopsutil/v3 v3.24.2 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/cast v1.9.2
github.com/spf13/cast v1.5.1
github.com/syndtr/goleveldb v1.0.0
github.com/valyala/fasttemplate v1.2.2
github.com/weppos/publicsuffix-go v0.50.1
github.com/weppos/publicsuffix-go v0.40.2
github.com/xanzy/go-gitlab v0.107.0
go.uber.org/multierr v1.11.0
golang.org/x/net v0.48.0
golang.org/x/oauth2 v0.30.0
golang.org/x/text v0.32.0
golang.org/x/net v0.39.0
golang.org/x/oauth2 v0.22.0
golang.org/x/text v0.24.0
gopkg.in/yaml.v2 v2.4.0
)
require (
carvel.dev/ytt v0.52.0
code.gitea.io/sdk/gitea v0.17.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.1
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.1.0
github.com/DataDog/gostackparse v0.7.0
github.com/DataDog/gostackparse v0.6.0
github.com/Masterminds/semver/v3 v3.2.1
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057
github.com/Mzack9999/goja v0.0.0-20250507184235-e46100e9c697
github.com/Mzack9999/goja_nodejs v0.0.0-20250507184139-66bcbf65c883
github.com/alexsnet/go-vnc v0.1.0
github.com/alitto/pond v1.9.2
github.com/antchfx/xmlquery v1.4.4
github.com/antchfx/xpath v1.3.5
github.com/antchfx/xmlquery v1.3.17
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2
github.com/aws/aws-sdk-go-v2 v1.36.5
github.com/aws/aws-sdk-go-v2/config v1.29.17
github.com/aws/aws-sdk-go-v2/credentials v1.17.70
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.82
github.com/aws/aws-sdk-go-v2/service/s3 v1.82.0
github.com/bytedance/sonic v1.14.0
github.com/aws/aws-sdk-go-v2 v1.19.0
github.com/aws/aws-sdk-go-v2/config v1.18.28
github.com/aws/aws-sdk-go-v2/credentials v1.13.27
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.72
github.com/aws/aws-sdk-go-v2/service/s3 v1.37.0
github.com/bytedance/sonic v1.12.8
github.com/cespare/xxhash v1.1.0
github.com/charmbracelet/glamour v0.10.0
github.com/charmbracelet/glamour v0.8.0
github.com/clbanning/mxj/v2 v2.7.0
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c
github.com/docker/go-units v0.5.0
github.com/dop251/goja v0.0.0-20240220182346-e401ed450204
github.com/fatih/structs v1.1.0
github.com/getkin/kin-openapi v0.132.0
github.com/go-git/go-git/v5 v5.16.2
github.com/go-ldap/ldap/v3 v3.4.11
github.com/go-pg/pg/v10 v10.15.0
github.com/go-sql-driver/mysql v1.9.3
github.com/getkin/kin-openapi v0.126.0
github.com/go-git/go-git/v5 v5.13.0
github.com/go-ldap/ldap/v3 v3.4.5
github.com/go-pg/pg v8.0.7+incompatible
github.com/go-sql-driver/mysql v1.7.1
github.com/goccy/go-json v0.10.5
github.com/google/uuid v1.6.0
github.com/h2non/filetype v1.1.3
github.com/invopop/yaml v0.3.1
github.com/jcmturner/gokrb5/v8 v8.4.4
github.com/kitabisa/go-ci v1.0.3
github.com/labstack/echo/v4 v4.13.4
github.com/labstack/echo/v4 v4.13.3
github.com/leslie-qiwa/flat v0.0.0-20230424180412-f9d1cf014baa
github.com/lib/pq v1.10.9
github.com/mattn/go-sqlite3 v1.14.28
github.com/maypok86/otter/v2 v2.2.1
github.com/mholt/archives v0.1.5
github.com/microsoft/go-mssqldb v1.9.2
github.com/ory/dockertest/v3 v3.12.0
github.com/praetorian-inc/fingerprintx v1.1.15
github.com/projectdiscovery/dsl v0.8.10
github.com/mattn/go-sqlite3 v1.14.22
github.com/mholt/archives v0.1.0
github.com/microsoft/go-mssqldb v1.6.0
github.com/ory/dockertest/v3 v3.10.0
github.com/praetorian-inc/fingerprintx v1.1.9
github.com/projectdiscovery/dsl v0.4.3
github.com/projectdiscovery/fasttemplate v0.0.2
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb
github.com/projectdiscovery/goflags v0.1.74
github.com/projectdiscovery/gologger v1.1.66
github.com/projectdiscovery/gologger v1.1.54
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/gozero v0.1.1-0.20251027191944-a4ea43320b81
github.com/projectdiscovery/httpx v1.7.4
github.com/projectdiscovery/mapcidr v1.1.97
github.com/projectdiscovery/gozero v0.0.3
github.com/projectdiscovery/httpx v1.7.0
github.com/projectdiscovery/mapcidr v1.1.34
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5
github.com/projectdiscovery/networkpolicy v0.1.33
github.com/projectdiscovery/ratelimit v0.0.82
github.com/projectdiscovery/rdap v0.9.0
github.com/projectdiscovery/ratelimit v0.0.80
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917
github.com/projectdiscovery/sarif v0.0.1
github.com/projectdiscovery/tlsx v1.2.2
github.com/projectdiscovery/uncover v1.2.0
github.com/projectdiscovery/useragent v0.0.106
github.com/projectdiscovery/utils v0.8.0
github.com/projectdiscovery/wappalyzergo v0.2.61
github.com/redis/go-redis/v9 v9.11.0
github.com/projectdiscovery/tlsx v1.1.9
github.com/projectdiscovery/uncover v1.0.10
github.com/projectdiscovery/useragent v0.0.100
github.com/projectdiscovery/utils v0.4.19
github.com/projectdiscovery/wappalyzergo v0.2.28
github.com/redis/go-redis/v9 v9.1.0
github.com/seh-msft/burpxml v1.0.1
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466
github.com/sijms/go-ora/v2 v2.9.0
github.com/stretchr/testify v1.11.1
github.com/stretchr/testify v1.10.0
github.com/tarunKoyalwar/goleak v0.0.0-20240429141123-0efa90dbdcf9
github.com/testcontainers/testcontainers-go v0.38.0
github.com/testcontainers/testcontainers-go/modules/mongodb v0.37.0
github.com/yassinebenaid/godump v0.11.1
github.com/zmap/zgrab2 v0.1.8
gitlab.com/gitlab-org/api/client-go v0.130.1
go.mongodb.org/mongo-driver v1.17.4
golang.org/x/term v0.38.0
github.com/yassinebenaid/godump v0.10.0
github.com/zmap/zgrab2 v0.1.8-0.20230806160807-97ba87c0e706
go.mongodb.org/mongo-driver v1.17.0
golang.org/x/term v0.31.0
gopkg.in/yaml.v3 v3.0.1
moul.io/http2curl v1.0.0
)
require (
aead.dev/minisign v0.2.0 // indirect
dario.cat/mergo v1.0.2 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
git.mills.io/prologic/smtpd v0.0.0-20210710122116-a525b76c287a // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
dario.cat/mergo v1.0.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect
github.com/BurntSushi/toml v1.3.2 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/PuerkitoBio/goquery v1.11.0 // indirect
github.com/STARRY-S/zip v0.2.3 // indirect
github.com/STARRY-S/zip v0.2.1 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/akrylysov/pogreb v0.10.2 // indirect
github.com/alecthomas/chroma/v2 v2.14.0 // indirect
github.com/alecthomas/kingpin/v2 v2.4.0 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.11 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.32 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.36 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.36 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.36 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.7.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.17 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.17 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.25.5 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.34.0 // indirect
github.com/aws/smithy-go v1.22.4 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.27 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.11 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.30 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.14.4 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/bits-and-blooms/bitset v1.13.0 // indirect
github.com/bits-and-blooms/bloom/v3 v3.5.0 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.6.1 // indirect
github.com/bodgit/sevenzip v1.6.0 // indirect
github.com/bodgit/windows v1.0.1 // indirect
github.com/brianvoe/gofakeit/v7 v7.2.1 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/bytedance/sonic/loader v0.3.0 // indirect
github.com/caddyserver/certmagic v0.19.2 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/censys/censys-sdk-go v0.19.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect
github.com/charmbracelet/lipgloss v1.1.1-0.20250404203927-76690c660834 // indirect
github.com/charmbracelet/x/ansi v0.8.0 // indirect
github.com/charmbracelet/x/cellbuf v0.0.13 // indirect
github.com/charmbracelet/x/exp/slice v0.0.0-20250327172914-2fdc97757edf // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/cheggaaa/pb/v3 v3.1.6 // indirect
github.com/bytedance/sonic/loader v0.2.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/charmbracelet/lipgloss v0.13.0 // indirect
github.com/charmbracelet/x/ansi v0.3.2 // indirect
github.com/cheggaaa/pb/v3 v3.1.4 // indirect
github.com/cloudflare/cfssl v1.6.4 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cloudflare/circl v1.3.8 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
github.com/cnf/structhash v0.0.0-20250313080605-df4c6cc74a9a // indirect
github.com/containerd/continuity v0.4.5 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/cyphar/filepath-securejoin v0.5.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/containerd/continuity v0.4.2 // indirect
github.com/cyphar/filepath-securejoin v0.2.5 // indirect
github.com/davidmz/go-pageant v1.0.2 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/djherbis/times v1.6.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/cli v27.4.1+incompatible // indirect
github.com/docker/docker v28.3.3+incompatible // indirect
github.com/docker/go-connections v0.6.0 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/ebitengine/purego v0.8.4 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/ericlagergren/decimal v0.0.0-20240411145413-00de7ca16731 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/docker/cli v24.0.5+incompatible // indirect
github.com/docker/docker v24.0.9+incompatible // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/felixge/fgprof v0.9.5 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/free5gc/util v1.0.5-0.20230511064842-2e120956883b // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gaissmai/bart v0.26.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect
github.com/gaissmai/bart v0.17.10 // indirect
github.com/geoffgarside/ber v1.1.0 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/gin-gonic/gin v1.9.1 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.4 // indirect
github.com/go-fed/httpsig v1.1.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-pg/zerochecker v0.2.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect
github.com/golang-sql/sqlexp v0.1.0 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/certificate-transparency-go v1.3.2 // indirect
github.com/google/certificate-transparency-go v1.1.4 // indirect
github.com/google/go-github/v30 v30.1.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/gosimple/slug v1.15.0 // indirect
github.com/gosimple/unidecode v1.0.1 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/go-version v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hbakhtiyor/strsim v0.0.0-20190107154042-4d2bbb273edf // indirect
github.com/hdm/jarm-go v0.0.7 // indirect
github.com/iangcarroll/cookiemonster v1.6.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/itchyny/timefmt-go v0.1.6 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
github.com/jcmturner/gofork v1.7.6 // indirect
github.com/jcmturner/rpc/v2 v2.0.3 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/k14s/starlark-go v0.0.0-20200720175618-3a5c849cc368 // indirect
github.com/kataras/jwt v0.1.10 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/libdns/libdns v0.2.1 // indirect
github.com/logrusorgru/aurora/v4 v4.0.0 // indirect
github.com/lor00x/goldap v0.0.0-20240304151906-8d785c64d1c8 // indirect
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
github.com/lufia/plan9stats v0.0.0-20250821153705-5981dea3221d // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mackerelio/go-osstat v0.2.4 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mholt/acmez v1.2.0 // indirect
github.com/microcosm-cc/bluemonday v1.0.27 // indirect
github.com/mikelolasagasti/xz v1.0.1 // indirect
github.com/minio/minlz v1.0.1 // indirect
github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826 // indirect
github.com/montanaflynn/stats v0.7.1 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/nwaples/rardecode/v2 v2.2.1 // indirect
github.com/oasdiff/yaml v0.0.0-20250309154309-f31be36b4037 // indirect
github.com/oasdiff/yaml3 v0.0.0-20250309153720-d2182401db90 // indirect
github.com/olekukonko/errors v1.1.0 // indirect
github.com/olekukonko/ll v0.0.9 // indirect
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a // indirect
github.com/nwaples/rardecode/v2 v2.0.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/opencontainers/runc v1.2.8 // indirect
github.com/openrdap/rdap v0.9.1 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/opencontainers/runc v1.1.14 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/perimeterx/marshmallow v1.1.5 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pjbgf/sha1cd v0.3.2 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/projectdiscovery/asnmap v1.1.1 // indirect
github.com/projectdiscovery/blackrock v0.0.1 // indirect
github.com/projectdiscovery/cdncheck v1.2.16 // indirect
github.com/projectdiscovery/cdncheck v1.1.17 // indirect
github.com/projectdiscovery/freeport v0.0.7 // indirect
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb // indirect
github.com/projectdiscovery/machineid v0.0.0-20250715113114-c77eb3567582 // indirect
github.com/refraction-networking/utls v1.7.1 // indirect
github.com/projectdiscovery/machineid v0.0.0-20240226150047-2e2c51e35983 // indirect
github.com/refraction-networking/utls v1.6.7 // indirect
github.com/sashabaranov/go-openai v1.37.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
github.com/shirou/gopsutil/v4 v4.25.7 // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/sorairolake/lzip-go v0.3.8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/skeema/knownhosts v1.3.0 // indirect
github.com/sorairolake/lzip-go v0.3.5 // indirect
github.com/therootcompany/xz v1.0.1 // indirect
github.com/tidwall/btree v1.7.0 // indirect
github.com/tidwall/buntdb v1.3.1 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/grect v0.1.4 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/rtred v0.1.2 // indirect
github.com/tidwall/tinyqueue v0.1.1 // indirect
github.com/tim-ywliu/nested-logrus-formatter v1.3.2 // indirect
github.com/tklauser/go-sysconf v0.3.15 // indirect
github.com/tklauser/numcpus v0.10.0 // indirect
github.com/tmthrgd/go-hex v0.0.0-20190904060850-447a3041c3bc // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/vmihailenco/bufpool v0.1.11 // indirect
github.com/vmihailenco/msgpack/v5 v5.3.4 // indirect
github.com/vmihailenco/tagparser v0.1.2 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
github.com/vulncheck-oss/go-exploit v1.51.0 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.2 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
github.com/ysmood/fetchup v0.2.3 // indirect
github.com/ysmood/got v0.40.0 // indirect
github.com/yuin/goldmark v1.7.13 // indirect
github.com/yuin/goldmark-emoji v1.0.5 // indirect
github.com/yuin/goldmark v1.7.4 // indirect
github.com/yuin/goldmark-emoji v1.0.3 // indirect
github.com/zcalusic/sysinfo v1.0.2 // indirect
github.com/zeebo/blake3 v0.2.3 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.uber.org/goleak v1.3.0 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/arch v0.3.0 // indirect
golang.org/x/sync v0.19.0 // indirect
mellium.im/sasl v0.3.2 // indirect
golang.org/x/sync v0.13.0 // indirect
gopkg.in/djherbis/times.v1 v1.3.0 // indirect
mellium.im/sasl v0.3.1 // indirect
)
require (
git.mills.io/prologic/smtpd v0.0.0-20210710122116-a525b76c287a // indirect
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 // indirect
github.com/PuerkitoBio/goquery v1.10.3 // indirect
github.com/akrylysov/pogreb v0.10.2 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/antchfx/xpath v1.2.4
github.com/aymerick/douceur v0.2.0 // indirect
github.com/caddyserver/certmagic v0.19.2 // indirect
github.com/cnf/structhash v0.0.0-20201127153200-e1b16c1ebc08 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/goburrow/cache v0.1.4 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
github.com/golang-jwt/jwt/v4 v4.5.1 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/uuid v1.6.0
github.com/gorilla/css v1.0.1 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
github.com/hdm/jarm-go v0.0.7 // indirect
github.com/itchyny/timefmt-go v0.1.5 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/libdns/libdns v0.2.1 // indirect
github.com/lor00x/goldap v0.0.0-20180618054307-a546dffdd1a3 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mholt/acmez v1.2.0 // indirect
github.com/microcosm-cc/bluemonday v1.0.27 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/projectdiscovery/blackrock v0.0.1 // indirect
github.com/projectdiscovery/networkpolicy v0.1.14
github.com/rivo/uniseg v0.4.7 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/trivago/tgo v1.0.7
github.com/ulikunitz/xz v0.5.12 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/ysmood/goob v0.4.0 // indirect
github.com/ysmood/gson v0.7.3 // indirect
github.com/ysmood/leakless v0.9.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
github.com/zmap/rc2 v0.0.0-20190804163417-abaa70531248 // indirect
github.com/zmap/zcrypto v0.0.0-20240512203510-0fef58d9a9db // indirect
go.etcd.io/bbolt v1.4.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.etcd.io/bbolt v1.3.10 // indirect
go.uber.org/zap v1.25.0 // indirect
goftp.io/server/v2 v2.0.1 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/exp v0.0.0-20250911091902-df9299821621
golang.org/x/mod v0.30.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.39.0
google.golang.org/protobuf v1.36.6 // indirect
golang.org/x/crypto v0.37.0 // indirect
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8
golang.org/x/mod v0.22.0 // indirect
golang.org/x/sys v0.32.0 // indirect
golang.org/x/time v0.8.0 // indirect
golang.org/x/tools v0.29.0
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/alecthomas/kingpin.v2 v2.2.6 // indirect
gopkg.in/corvus-ch/zbase32.v1 v1.0.0 // indirect
)
require (
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v1.1.3 // indirect
github.com/alecthomas/chroma v0.10.0
github.com/go-echarts/go-echarts/v2 v2.6.0
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.5 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.35 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.36 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.29 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.12.13 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.19.3 // indirect
github.com/aws/smithy-go v1.13.5 // indirect
github.com/dop251/goja_nodejs v0.0.0-20230821135201-94e508132562
github.com/emirpasic/gods v1.18.1 // indirect
github.com/go-echarts/go-echarts/v2 v2.3.3
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.6.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jcmturner/gokrb5/v8 v8.4.4
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
)
+555 -500
View File
File diff suppressed because it is too large. Load diff
-38
View File
@@ -1,38 +0,0 @@
id: fuzz-body
info:
name: fuzzing error sqli payloads in http req body
author: pdteam
severity: info
description: |
This template attempts to find SQL injection vulnerabilities by fuzzing http body
It automatically handles and parses json,xml,multipart form and x-www-form-urlencoded data
and performs fuzzing on the value of every key
http:
- pre-condition:
- type: dsl
dsl:
- method != "GET"
- method != "HEAD"
condition: and
payloads:
injection:
- "'"
- "\""
- ";"
fuzzing:
- part: body
type: postfix
mode: single
fuzz:
- '{{injection}}'
stop-at-first-match: true
matchers:
- type: word
words:
- "unrecognized token:"
- "null"
@@ -1,27 +0,0 @@
id: pw-echo
info:
name: PowerShell Echo Test
author: pdteam
severity: info
description: Tests PowerShell execution with an echo-like operation.
tags: test,powershell,echo
self-contained: true
code:
- engine:
- pwsh
- powershell
- powershell.exe
args:
- -ExecutionPolicy
- Bypass
pattern: "*.ps1"
source: |
Write-Output "test-output-success"
matchers:
- type: word
words:
- "test-output-success"
@@ -1,27 +0,0 @@
id: signed-python-code-in-virtual-env
info:
name: signed-python-code-in-virtual-env
author: pdteam
severity: info
tags: code
description: |
signed python code in virtual environment
code:
- engine:
- sh
- bash
sandbox:
working-dir: /tmp
image: python:3.14
source: |
#!/usr/bin/env python3
import sys
print("hello from python virtual code")
matchers:
- type: word
words:
- "hello from python virtual code"
@@ -1,25 +0,0 @@
id: signed-code-in-virtual-env
info:
name: signed-code-in-virtual-env
author: pdteam
severity: info
tags: code
description: |
signed code in virtual environment
code:
- engine:
- sh
- bash
sandbox:
working-dir: /tmp
image: ubuntu:latest
source: |
echo "hello from sh virtual code"
matchers:
- type: word
words:
- "hello from sh virtual code"
# digest: 4a0a00473045022100a2a71c423b72e600ac2d78209482b48591924157d49e2dfb7767445b0073e92b022009a6365e84247e268c7ffcb2f9ed424529dada6f087d5f21400ab0b2a822ca56:4a3eb6b4988d95847d4203be25ed1d46
@@ -1,27 +0,0 @@
id: constants-with-threads
info:
name: Constants with Threads
author: pdteam
severity: info
description: |
Test that constants are properly resolved when using threads mode.
constants:
api_key: "supersecretkey123"
api_version: "v2"
http:
- method: GET
path:
- "{{BaseURL}}/api/{{api_version}}"
threads: 5
headers:
X-API-Key: "{{api_key}}"
matchers:
- type: word
words:
- "supersecretkey123"
- "v2"
condition: and
@@ -1,30 +0,0 @@
id: race-with-variables
info:
name: Race Condition with Variables
author: pdteam
severity: info
description: |
Test that variables and constants are properly resolved in race mode.
variables:
random_id: "{{rand_base(8)}}"
constants:
api_key: "racekey123"
http:
- raw:
- |
GET /race HTTP/1.1
Host: {{Hostname}}
X-Request-Id: {{random_id}}
X-API-Key: {{api_key}}
race: true
race_count: 3
matchers:
- type: word
words:
- "racekey123"
@@ -1,38 +0,0 @@
id: variables-threads-previous
info:
name: Variables with Threads and Previous Request Data
author: pdteam
severity: info
description: |
Test that variables can reference data extracted from previous requests
when using threads mode (parallel execution).
variables:
auth_header: "Bearer {{extracted_token}}"
http:
- method: GET
path:
- "{{BaseURL}}/login"
extractors:
- type: regex
name: extracted_token
part: body
regex:
- 'token=([a-z0-9]+)'
group: 1
internal: true
- method: GET
path:
- "{{BaseURL}}/api"
threads: 5
headers:
Authorization: "{{auth_header}}"
matchers:
- type: word
words:
- "Bearer secret123"
@@ -1,28 +0,0 @@
id: multi-ports
info:
name: Multi Ports - Detection
author: pdteam
severity: info
description: |
Multi Ports template for testing
metadata:
max-request: 1
tags: js,detect,multi-ports,enum,network
javascript:
- pre-condition: |
isPortOpen(Host,Port);
code: |
var m = require("nuclei/ssh");
var c = m.SSHClient();
var response = c.ConnectSSHInfoMode(Host, Port);
Export(response);
args:
Host: "{{Host}}"
Port: "2222,22" # Port 22 should match
extractors:
- type: json
json:
- '.UserAuth'
@@ -1,23 +0,0 @@
id: mysql-connect
info:
name: MySQL Connect Test
author: pdteam
severity: high
javascript:
- pre-condition: |
isPortOpen(Host, Port)
code: |
const mysql = require('nuclei/mysql');
const client = new mysql.MySQLClient;
success = client.Connect(Host, Port, User, Pass);
args:
Host: "{{Host}}"
Port: "3306"
User: "root"
Pass: "secret"
matchers:
- type: dsl
dsl:
- "success == true"
@@ -1,23 +0,0 @@
id: javascript-no-port-args
info:
name: JavaScript Template Without Port Args
author: dwisiswant0
severity: info
description: |
Test backwards compatibility for JavaScript templates without Port in args.
Templates should execute even when Port arg is not explicitly specified.
javascript:
- code: |
// Simple JavaScript code that does not require Port
let result = "executed";
Export(result);
args:
TestArg: "test-value"
matchers:
- type: dsl
dsl:
- success == true
@@ -1,47 +0,0 @@
id: postgres-pass-brute
info:
name: PostgreSQL Password Bruteforce
author: pdteam
severity: high
description: |
This template bruteforces passwords for protected PostgreSQL instances.
If PostgreSQL is not protected with password, it is also matched.
metadata:
shodan-query: product:"PostgreSQL"
tags: js,network,postgresql,authentication
javascript:
- pre-condition: |
isPortOpen(Host,Port)
code: |
const postgres = require('nuclei/postgres');
const client = new postgres.PGClient;
success = client.Connect(Host, Port, User, Pass);
args:
Host: "{{Host}}"
Port: "5432"
User: "{{usernames}}"
Pass: "{{passwords}}"
attack: clusterbomb
payloads:
usernames:
- postgres
- admin
- root
passwords:
- ""
- postgres
- password
- admin
- root
stop-at-first-match: true
matchers:
- type: dsl
dsl:
- "success == true"
@@ -1,38 +0,0 @@
id: vnc-password-test
info:
name: VNC Password Authentication Test
author: pdteam
severity: high
description: |
Tests VNC authentication with correct and incorrect passwords.
metadata:
shodan-query: product:"vnc"
tags: js,network,vnc,authentication
javascript:
- pre-condition: |
isPortOpen(Host,Port)
code: |
let vnc = require('nuclei/vnc');
let client = new vnc.VNCClient();
client.Connect(Host, Port, Password);
args:
Host: "{{Host}}"
Port: "5900"
Password: "{{passwords}}"
payloads:
passwords:
- ""
- root
- password
- admin
- mysecret
stop-at-first-match: true
matchers:
- type: dsl
dsl:
- "success == true"
+1 -1
View File
@@ -1,3 +1,3 @@
## keys
the keys stored here especially `ci-private-key.pem` and `ci.crt` are used in integration tests to test template signing and verification functionality introduced in nuclei v3
the keys stored here especially `ci-private-key.pem` and `ci.crt` are used in integration tests to test template signing and verfication functionality introduced in nuclei v3
+24 -30
View File
@@ -19,7 +19,7 @@ import (
"github.com/projectdiscovery/retryablehttp-go"
pdcpauth "github.com/projectdiscovery/utils/auth/pdcp"
"github.com/projectdiscovery/utils/env"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
unitutils "github.com/projectdiscovery/utils/unit"
updateutils "github.com/projectdiscovery/utils/update"
urlutil "github.com/projectdiscovery/utils/url"
@@ -55,11 +55,10 @@ type UploadWriter struct {
scanName string
counter atomic.Int32
TeamID string
Logger *gologger.Logger
}
// NewUploadWriter creates a new upload writer
func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpauth.PDCPCredentials) (*UploadWriter, error) {
func NewUploadWriter(ctx context.Context, creds *pdcpauth.PDCPCredentials) (*UploadWriter, error) {
if creds == nil {
return nil, fmt.Errorf("no credentials provided")
}
@@ -67,7 +66,6 @@ func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpau
creds: creds,
done: make(chan struct{}, 1),
TeamID: NoneTeamID,
Logger: logger,
}
var err error
reader, writer := io.Pipe()
@@ -77,11 +75,11 @@ func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpau
output.WithJson(true, true),
)
if err != nil {
return nil, errkit.Wrap(err, "could not create output writer")
return nil, errorutil.NewWithErr(err).Msgf("could not create output writer")
}
tmp, err := urlutil.Parse(creds.Server)
if err != nil {
return nil, errkit.Wrap(err, "could not parse server url")
return nil, errorutil.NewWithErr(err).Msgf("could not parse server url")
}
tmp.Path = uploadEndpoint
tmp.Update()
@@ -104,7 +102,7 @@ func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpau
// SetScanID sets the scan id for the upload writer
func (u *UploadWriter) SetScanID(id string) error {
if !xidRegex.MatchString(id) {
gologger.Warning().Msgf("invalid asset id provided (unknown xid format): %s", id)
return fmt.Errorf("invalid scan id provided")
}
u.scanID = id
return nil
@@ -129,9 +127,7 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
// continuously read from the reader and send to channel
go func() {
defer func() {
_ = r.Close()
}()
defer r.Close()
defer close(ch)
for {
data, err := reader.ReadString('\n')
@@ -149,9 +145,9 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
close(u.done)
// if no scanid is generated no results were uploaded
if u.scanID == "" {
u.Logger.Verbose().Msgf("Scan results upload to cloud skipped, no results found to upload")
gologger.Verbose().Msgf("Scan results upload to cloud skipped, no results found to upload")
} else {
u.Logger.Info().Msgf("%v Scan results uploaded to cloud, you can view scan results at %v", u.counter.Load(), getScanDashBoardURL(u.scanID, u.TeamID))
gologger.Info().Msgf("%v Scan results uploaded to cloud, you can view scan results at %v", u.counter.Load(), getScanDashBoardURL(u.scanID, u.TeamID))
}
}()
// temporary buffer to store the results
@@ -164,7 +160,7 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
// flush before exit
if buff.Len() > 0 {
if err := u.uploadChunk(buff); err != nil {
u.Logger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
gologger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
}
}
return
@@ -172,14 +168,14 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
// flush the buffer
if buff.Len() > 0 {
if err := u.uploadChunk(buff); err != nil {
u.Logger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
gologger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
}
}
case line, ok := <-ch:
if !ok {
if buff.Len() > 0 {
if err := u.uploadChunk(buff); err != nil {
u.Logger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
gologger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
}
}
return
@@ -187,7 +183,7 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
if buff.Len()+len(line) > MaxChunkSize {
// flush existing buffer
if err := u.uploadChunk(buff); err != nil {
u.Logger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
gologger.Error().Msgf("Failed to upload scan results on cloud: %v", err)
}
} else {
buff.WriteString(line)
@@ -199,37 +195,35 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
// uploadChunk uploads a chunk of data to the server
func (u *UploadWriter) uploadChunk(buff *bytes.Buffer) error {
if err := u.upload(buff.Bytes()); err != nil {
return errkit.Wrap(err, "could not upload chunk")
return errorutil.NewWithErr(err).Msgf("could not upload chunk")
}
// if successful, reset the buffer
buff.Reset()
// log in verbose mode
u.Logger.Warning().Msgf("Uploaded results chunk, you can view scan results at %v", getScanDashBoardURL(u.scanID, u.TeamID))
gologger.Warning().Msgf("Uploaded results chunk, you can view scan results at %v", getScanDashBoardURL(u.scanID, u.TeamID))
return nil
}
func (u *UploadWriter) upload(data []byte) error {
req, err := u.getRequest(data)
if err != nil {
return errkit.Wrap(err, "could not create upload request")
return errorutil.NewWithErr(err).Msgf("could not create upload request")
}
resp, err := u.client.Do(req)
if err != nil {
return errkit.Wrap(err, "could not upload results")
return errorutil.NewWithErr(err).Msgf("could not upload results")
}
defer func() {
_ = resp.Body.Close()
}()
defer resp.Body.Close()
bin, err := io.ReadAll(resp.Body)
if err != nil {
return errkit.Wrap(err, "could not get id from response")
return errorutil.NewWithErr(err).Msgf("could not get id from response")
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("could not upload results got status code %v on %v", resp.StatusCode, resp.Request.URL.String())
}
var uploadResp uploadResponse
if err := json.Unmarshal(bin, &uploadResp); err != nil {
return errkit.Wrap(err, fmt.Sprintf("could not unmarshal response got %v", string(bin)))
return errorutil.NewWithErr(err).Msgf("could not unmarshal response got %v", string(bin))
}
if uploadResp.ID != "" && u.scanID == "" {
u.scanID = uploadResp.ID
@@ -254,15 +248,15 @@ func (u *UploadWriter) getRequest(bin []byte) (*retryablehttp.Request, error) {
}
req, err := retryablehttp.NewRequest(method, url, bytes.NewReader(bin))
if err != nil {
return nil, errkit.Wrap(err, "could not create cloud upload request")
return nil, errorutil.NewWithErr(err).Msgf("could not create cloud upload request")
}
// add pdtm meta params
req.Params.Merge(updateutils.GetpdtmParams(config.Version))
req.URL.Params.Merge(updateutils.GetpdtmParams(config.Version))
// if it is upload endpoint also include name if it exists
if u.scanName != "" && req.Path == uploadEndpoint {
req.Params.Add("name", u.scanName)
if u.scanName != "" && req.URL.Path == uploadEndpoint {
req.URL.Params.Add("name", u.scanName)
}
req.Update()
req.URL.Update()
req.Header.Set(pdcpauth.ApiKeyHeaderName, u.creds.APIKey)
if u.TeamID != NoneTeamID && u.TeamID != "" {
+3 -3
View File
@@ -47,7 +47,7 @@ func DoHealthCheck(options *types.Options) string {
}
c4, err := net.Dial("tcp4", "scanme.sh:80")
if err == nil && c4 != nil {
_ = c4.Close()
c4.Close()
}
testResult = "Ok"
if err != nil {
@@ -56,7 +56,7 @@ func DoHealthCheck(options *types.Options) string {
test.WriteString(fmt.Sprintf("IPv4 connectivity to scanme.sh:80 => %s\n", testResult))
c6, err := net.Dial("tcp6", "scanme.sh:80")
if err == nil && c6 != nil {
_ = c6.Close()
c6.Close()
}
testResult = "Ok"
if err != nil {
@@ -65,7 +65,7 @@ func DoHealthCheck(options *types.Options) string {
test.WriteString(fmt.Sprintf("IPv6 connectivity to scanme.sh:80 => %s\n", testResult))
u4, err := net.Dial("udp4", "scanme.sh:53")
if err == nil && u4 != nil {
_ = u4.Close()
u4.Close()
}
testResult = "Ok"
if err != nil {
+5 -11
View File
@@ -2,11 +2,11 @@ package runner
import (
"context"
"fmt"
"sync/atomic"
"time"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/hmap/store/hybrid"
"github.com/projectdiscovery/httpx/common/httpx"
"github.com/projectdiscovery/nuclei/v3/pkg/input/provider"
@@ -28,7 +28,7 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
// currently http probing for input mode types is not supported
return hm, nil
}
r.Logger.Info().Msgf("Running httpx on input host")
gologger.Info().Msgf("Running httpx on input host")
httpxOptions := httpx.DefaultOptions
if r.options.AliveHttpProxy != "" {
@@ -38,13 +38,7 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
}
httpxOptions.RetryMax = r.options.Retries
httpxOptions.Timeout = time.Duration(r.options.Timeout) * time.Second
dialers := protocolstate.GetDialersWithId(r.options.ExecutionId)
if dialers == nil {
return nil, fmt.Errorf("dialers not initialized for %s", r.options.ExecutionId)
}
httpxOptions.NetworkPolicy = dialers.NetworkPolicy
httpxOptions.NetworkPolicy = protocolstate.NetworkPolicy
httpxClient, err := httpx.New(&httpxOptions)
if err != nil {
return nil, errors.Wrap(err, "could not create httpx client")
@@ -63,7 +57,7 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
if r.options.ProbeConcurrency > 0 && swg.Size != r.options.ProbeConcurrency {
if err := swg.Resize(context.Background(), r.options.ProbeConcurrency); err != nil {
r.Logger.Error().Msgf("Could not resize workpool: %s\n", err)
gologger.Error().Msgf("Could not resize workpool: %s\n", err)
}
}
@@ -80,6 +74,6 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
})
swg.Wait()
r.Logger.Info().Msgf("Found %d URL from httpx", count.Load())
gologger.Info().Msgf("Found %d URL from httpx", count.Load())
return hm, nil
}
+7 -8
View File
@@ -7,7 +7,6 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/authx"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
@@ -18,22 +17,22 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/scan"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/utils/env"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
)
type AuthLazyFetchOptions struct {
TemplateStore *loader.Store
ExecOpts *protocols.ExecutorOptions
ExecOpts protocols.ExecutorOptions
OnError func(error)
}
// GetAuthTmplStore create new loader for loading auth templates
func GetAuthTmplStore(opts *types.Options, catalog catalog.Catalog, execOpts *protocols.ExecutorOptions) (*loader.Store, error) {
func GetAuthTmplStore(opts types.Options, catalog catalog.Catalog, execOpts protocols.ExecutorOptions) (*loader.Store, error) {
tmpls := []string{}
for _, file := range opts.SecretsFile {
data, err := authx.GetTemplatePathsFromSecretFile(file)
if err != nil {
return nil, errkit.Wrap(err, "failed to get template paths from secrets file")
return nil, errorutil.NewWithErr(err).Msgf("failed to get template paths from secrets file")
}
tmpls = append(tmpls, data...)
}
@@ -55,11 +54,11 @@ func GetAuthTmplStore(opts *types.Options, catalog catalog.Catalog, execOpts *pr
opts.Protocols = nil
opts.ExcludeProtocols = nil
opts.IncludeConditions = nil
cfg := loader.NewConfig(opts, catalog, execOpts)
cfg := loader.NewConfig(&opts, catalog, execOpts)
cfg.StoreId = loader.AuthStoreId
store, err := loader.New(cfg)
if err != nil {
return nil, errkit.Wrap(err, "failed to initialize dynamic auth templates store")
return nil, errorutil.NewWithErr(err).Msgf("failed to initialize dynamic auth templates store")
}
return store, nil
}
@@ -69,7 +68,7 @@ func GetLazyAuthFetchCallback(opts *AuthLazyFetchOptions) authx.LazyFetchSecret
return func(d *authx.Dynamic) error {
tmpls := opts.TemplateStore.LoadTemplates([]string{d.TemplatePath})
if len(tmpls) == 0 {
return fmt.Errorf("%w for path: %s", disk.ErrNoTemplatesFound, d.TemplatePath)
return fmt.Errorf("no templates found for path: %s", d.TemplatePath)
}
if len(tmpls) > 1 {
return fmt.Errorf("multiple templates found for path: %s", d.TemplatePath)
+29 -34
View File
@@ -31,6 +31,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/utils/yaml"
fileutil "github.com/projectdiscovery/utils/file"
"github.com/projectdiscovery/utils/generic"
logutil "github.com/projectdiscovery/utils/log"
stringsutil "github.com/projectdiscovery/utils/strings"
)
@@ -39,8 +40,6 @@ const (
DefaultDumpTrafficOutputFolder = "output"
)
var validateOptions = validator.New()
func ConfigureOptions() error {
// with FileStringSliceOptions, FileNormalizedStringSliceOptions, FileCommaSeparatedStringSliceOptions
// if file has the extension `.yaml` or `.json` we consider those as strings and not files to be read
@@ -72,17 +71,17 @@ func ParseOptions(options *types.Options) {
vardump.Limit = options.VarDumpLimit
}
if options.ShowActions {
options.Logger.Info().Msgf("Showing available headless actions: ")
gologger.Info().Msgf("Showing available headless actions: ")
for action := range engine.ActionStringToAction {
options.Logger.Print().Msgf("\t%s", action)
gologger.Print().Msgf("\t%s", action)
}
os.Exit(0)
}
defaultProfilesPath := filepath.Join(config.DefaultConfig.GetTemplateDir(), "profiles")
if options.ListTemplateProfiles {
options.Logger.Print().Msgf(
"Listing available %v nuclei template profiles for %v",
gologger.Print().Msgf(
"\nListing available %v nuclei template profiles for %v",
config.DefaultConfig.TemplateVersion,
config.DefaultConfig.TemplatesDirectory,
)
@@ -94,23 +93,23 @@ func ParseOptions(options *types.Options) {
return nil
}
if profileRelPath, err := filepath.Rel(templatesRootDir, iterItem); err == nil {
options.Logger.Print().Msgf("%s (%s)\n", profileRelPath, strings.TrimSuffix(filepath.Base(iterItem), ext))
gologger.Print().Msgf("%s (%s)\n", profileRelPath, strings.TrimSuffix(filepath.Base(iterItem), ext))
}
return nil
})
if err != nil {
options.Logger.Error().Msgf("%s\n", err)
gologger.Error().Msgf("%s\n", err)
}
os.Exit(0)
}
if options.StoreResponseDir != DefaultDumpTrafficOutputFolder && !options.StoreResponse {
options.Logger.Debug().Msgf("Store response directory specified, enabling \"store-resp\" flag automatically\n")
gologger.Debug().Msgf("Store response directory specified, enabling \"store-resp\" flag automatically\n")
options.StoreResponse = true
}
// Validate the options passed by the user and if any
// invalid options have been used, exit.
if err := ValidateOptions(options); err != nil {
options.Logger.Fatal().Msgf("Program exiting: %s\n", err)
gologger.Fatal().Msgf("Program exiting: %s\n", err)
}
// Load the resolvers if user asked for them
@@ -118,12 +117,12 @@ func ParseOptions(options *types.Options) {
err := protocolinit.Init(options)
if err != nil {
options.Logger.Fatal().Msgf("Could not initialize protocols: %s\n", err)
gologger.Fatal().Msgf("Could not initialize protocols: %s\n", err)
}
// Set GitHub token in env variable. runner.getGHClientWithToken() reads token from env
if options.GitHubToken != "" && os.Getenv("GITHUB_TOKEN") != options.GitHubToken {
_ = os.Setenv("GITHUB_TOKEN", options.GitHubToken)
os.Setenv("GITHUB_TOKEN", options.GitHubToken)
}
if options.UncoverQuery != nil {
@@ -140,7 +139,8 @@ func ParseOptions(options *types.Options) {
// validateOptions validates the configuration options passed
func ValidateOptions(options *types.Options) error {
if err := validateOptions.Struct(options); err != nil {
validate := validator.New()
if err := validate.Struct(options); err != nil {
if _, ok := err.(*validator.InvalidValidationError); ok {
return err
}
@@ -169,7 +169,7 @@ func ValidateOptions(options *types.Options) error {
return err
}
if options.Validate {
validateTemplatePaths(options.Logger, config.DefaultConfig.TemplatesDirectory, options.Templates, options.Workflows)
validateTemplatePaths(config.DefaultConfig.TemplatesDirectory, options.Templates, options.Workflows)
}
if options.DAST {
if err := validateDASTOptions(options); err != nil {
@@ -182,7 +182,7 @@ func ValidateOptions(options *types.Options) error {
if generic.EqualsAny("", options.ClientCertFile, options.ClientKeyFile, options.ClientCAFile) {
return errors.New("if a client certification option is provided, then all three must be provided")
}
validateCertificatePaths(options.Logger, options.ClientCertFile, options.ClientKeyFile, options.ClientCAFile)
validateCertificatePaths(options.ClientCertFile, options.ClientKeyFile, options.ClientCAFile)
}
// Verify AWS secrets are passed if a S3 template bucket is passed
if options.AwsBucketName != "" && options.UpdateTemplates && !options.AwsTemplateDisableDownload {
@@ -304,9 +304,7 @@ func createReportingOptions(options *types.Options) (*reporting.Options, error)
if err != nil {
return nil, errors.Wrap(err, "could not open reporting config file")
}
defer func() {
_ = file.Close()
}()
defer file.Close()
if err := yaml.DecodeAndValidate(file, reportingOptions); err != nil {
return nil, errors.Wrap(err, "could not parse reporting config file")
@@ -344,33 +342,32 @@ func createReportingOptions(options *types.Options) (*reporting.Options, error)
}
reportingOptions.OmitRaw = options.OmitRawRequests
reportingOptions.ExecutionId = options.ExecutionId
return reportingOptions, nil
}
// configureOutput configures the output logging levels to be displayed on the screen
func configureOutput(options *types.Options) {
if options.NoColor {
options.Logger.SetFormatter(formatter.NewCLI(true))
gologger.DefaultLogger.SetFormatter(formatter.NewCLI(true))
}
// If the user desires verbose output, show verbose output
if options.Debug || options.DebugRequests || options.DebugResponse {
options.Logger.SetMaxLevel(levels.LevelDebug)
gologger.DefaultLogger.SetMaxLevel(levels.LevelDebug)
}
// Debug takes precedence before verbose
// because debug is a lower logging level.
if options.Verbose || options.Validate {
options.Logger.SetMaxLevel(levels.LevelVerbose)
gologger.DefaultLogger.SetMaxLevel(levels.LevelVerbose)
}
if options.NoColor {
options.Logger.SetFormatter(formatter.NewCLI(true))
gologger.DefaultLogger.SetFormatter(formatter.NewCLI(true))
}
if options.Silent {
options.Logger.SetMaxLevel(levels.LevelSilent)
gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent)
}
// disable standard logger (ref: https://github.com/golang/go/issues/19895)
// logutil.DisableDefaultLogger()
logutil.DisableDefaultLogger()
}
// loadResolvers loads resolvers from both user-provided flags and file
@@ -381,11 +378,9 @@ func loadResolvers(options *types.Options) {
file, err := os.Open(options.ResolversFile)
if err != nil {
options.Logger.Fatal().Msgf("Could not open resolvers file: %s\n", err)
gologger.Fatal().Msgf("Could not open resolvers file: %s\n", err)
}
defer func() {
_ = file.Close()
}()
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
@@ -401,7 +396,7 @@ func loadResolvers(options *types.Options) {
}
}
func validateTemplatePaths(logger *gologger.Logger, templatesDirectory string, templatePaths, workflowPaths []string) {
func validateTemplatePaths(templatesDirectory string, templatePaths, workflowPaths []string) {
allGivenTemplatePaths := append(templatePaths, workflowPaths...)
for _, templatePath := range allGivenTemplatePaths {
if templatesDirectory != templatePath && filepath.IsAbs(templatePath) {
@@ -409,7 +404,7 @@ func validateTemplatePaths(logger *gologger.Logger, templatesDirectory string, t
if err == nil && fileInfo.IsDir() {
relativizedPath, err2 := filepath.Rel(templatesDirectory, templatePath)
if err2 != nil || (len(relativizedPath) >= 2 && relativizedPath[:2] == "..") {
logger.Warning().Msgf("The given path (%s) is outside the default template directory path (%s)! "+
gologger.Warning().Msgf("The given path (%s) is outside the default template directory path (%s)! "+
"Referenced sub-templates with relative paths in workflows will be resolved against the default template directory.", templatePath, templatesDirectory)
break
}
@@ -418,12 +413,12 @@ func validateTemplatePaths(logger *gologger.Logger, templatesDirectory string, t
}
}
func validateCertificatePaths(logger *gologger.Logger, certificatePaths ...string) {
func validateCertificatePaths(certificatePaths ...string) {
for _, certificatePath := range certificatePaths {
if !fileutil.FileExists(certificatePath) {
// The provided path to the PEM certificate does not exist for the client authentication. As this is
// required for successful authentication, log and return an error
logger.Fatal().Msgf("The given path (%s) to the certificate does not exist!", certificatePath)
gologger.Fatal().Msgf("The given path (%s) to the certificate does not exist!", certificatePath)
break
}
}
@@ -450,7 +445,7 @@ func readEnvInputVars(options *types.Options) {
// Attempt to convert the repo ID to an integer
repoIDInt, err := strconv.Atoi(repoID)
if err != nil {
options.Logger.Warning().Msgf("Invalid GitLab template repository ID: %s", repoID)
gologger.Warning().Msgf("Invalid GitLab template repository ID: %s", repoID)
continue
}
+9 -11
View File
@@ -7,8 +7,9 @@ import (
"os"
"strings"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/utils/errkit"
errorutil "github.com/projectdiscovery/utils/errors"
fileutil "github.com/projectdiscovery/utils/file"
proxyutils "github.com/projectdiscovery/utils/proxy"
)
@@ -29,9 +30,7 @@ func loadProxyServers(options *types.Options) error {
if err != nil {
return fmt.Errorf("could not open proxy file: %w", err)
}
defer func() {
_ = file.Close()
}()
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
proxy := scanner.Text()
@@ -50,18 +49,17 @@ func loadProxyServers(options *types.Options) error {
}
proxyURL, err := url.Parse(aliveProxy)
if err != nil {
return errkit.Wrapf(err, "failed to parse proxy got %v", err)
return errorutil.WrapfWithNil(err, "failed to parse proxy got %v", err)
}
if options.ProxyInternal {
_ = os.Setenv(HTTP_PROXY_ENV, proxyURL.String())
os.Setenv(HTTP_PROXY_ENV, proxyURL.String())
}
switch proxyURL.Scheme {
case proxyutils.HTTP, proxyutils.HTTPS:
options.Logger.Verbose().Msgf("Using %s as proxy server", proxyURL.String())
if proxyURL.Scheme == proxyutils.HTTP || proxyURL.Scheme == proxyutils.HTTPS {
gologger.Verbose().Msgf("Using %s as proxy server", proxyURL.String())
options.AliveHttpProxy = proxyURL.String()
case proxyutils.SOCKS5:
} else if proxyURL.Scheme == proxyutils.SOCKS5 {
options.AliveSocksProxy = proxyURL.String()
options.Logger.Verbose().Msgf("Using %s as socket proxy server", proxyURL.String())
gologger.Verbose().Msgf("Using %s as socket proxy server", proxyURL.String())
}
return nil
}
+69 -91
View File
@@ -10,7 +10,6 @@ import (
"sync/atomic"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/internal/pdcp"
"github.com/projectdiscovery/nuclei/v3/internal/server"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider"
@@ -33,6 +32,7 @@ import (
"github.com/pkg/errors"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/internal/colorizer"
"github.com/projectdiscovery/nuclei/v3/internal/httpapi"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog"
@@ -95,7 +95,6 @@ type Runner struct {
inputProvider provider.InputProvider
fuzzFrequencyCache *frequency.Tracker
httpStats *outputstats.Tracker
Logger *gologger.Logger
//general purpose temporary directory
tmpDir string
@@ -109,11 +108,10 @@ type Runner struct {
func New(options *types.Options) (*Runner, error) {
runner := &Runner{
options: options,
Logger: options.Logger,
}
if options.HealthCheck {
runner.Logger.Print().Msgf("%s\n", DoHealthCheck(options))
gologger.Print().Msgf("%s\n", DoHealthCheck(options))
os.Exit(0)
}
@@ -121,14 +119,14 @@ func New(options *types.Options) (*Runner, error) {
if config.DefaultConfig.CanCheckForUpdates() {
if err := installer.NucleiVersionCheck(); err != nil {
if options.Verbose || options.Debug {
runner.Logger.Error().Msgf("nuclei version check failed got: %s\n", err)
gologger.Error().Msgf("nuclei version check failed got: %s\n", err)
}
}
// check for custom template updates and update if available
ctm, err := customtemplates.NewCustomTemplatesManager(options)
if err != nil {
runner.Logger.Error().Label("custom-templates").Msgf("Failed to create custom templates manager: %s\n", err)
gologger.Error().Label("custom-templates").Msgf("Failed to create custom templates manager: %s\n", err)
}
// Check for template updates and update if available.
@@ -138,15 +136,15 @@ func New(options *types.Options) (*Runner, error) {
DisablePublicTemplates: options.PublicTemplateDisableDownload,
}
if err := tm.FreshInstallIfNotExists(); err != nil {
runner.Logger.Warning().Msgf("failed to install nuclei templates: %s\n", err)
gologger.Warning().Msgf("failed to install nuclei templates: %s\n", err)
}
if err := tm.UpdateIfOutdated(); err != nil {
runner.Logger.Warning().Msgf("failed to update nuclei templates: %s\n", err)
gologger.Warning().Msgf("failed to update nuclei templates: %s\n", err)
}
if config.DefaultConfig.NeedsIgnoreFileUpdate() {
if err := installer.UpdateIgnoreFile(); err != nil {
runner.Logger.Warning().Msgf("failed to update nuclei ignore file: %s\n", err)
gologger.Warning().Msgf("failed to update nuclei ignore file: %s\n", err)
}
}
@@ -154,7 +152,7 @@ func New(options *types.Options) (*Runner, error) {
// we automatically check for updates unless explicitly disabled
// this print statement is only to inform the user that there are no updates
if !config.DefaultConfig.NeedsTemplateUpdate() {
runner.Logger.Info().Msgf("No new updates found for nuclei templates")
gologger.Info().Msgf("No new updates found for nuclei templates")
}
// manually trigger update of custom templates
if ctm != nil {
@@ -163,25 +161,20 @@ func New(options *types.Options) (*Runner, error) {
}
}
if op, ok := options.Parser.(*templates.Parser); ok {
// Enable passing in an existing parser instance
// This uses a type assertion to avoid an import loop
runner.parser = op
} else {
parser := templates.NewParser()
if options.Validate {
parser.ShouldValidate = true
}
// TODO: refactor to pass options reference globally without cycles
parser.NoStrictSyntax = options.NoStrictSyntax
runner.parser = parser
parser := templates.NewParser()
if options.Validate {
parser.ShouldValidate = true
}
// TODO: refactor to pass options reference globally without cycles
parser.NoStrictSyntax = options.NoStrictSyntax
runner.parser = parser
yaml.StrictSyntax = !options.NoStrictSyntax
if options.Headless {
if engine.MustDisableSandbox() {
runner.Logger.Warning().Msgf("The current platform and privileged user will run the browser without sandbox\n")
gologger.Warning().Msgf("The current platform and privileged user will run the browser without sandbox\n")
}
browser, err := engine.New(options)
if err != nil {
@@ -233,11 +226,11 @@ func New(options *types.Options) (*Runner, error) {
if options.HttpApiEndpoint != "" {
apiServer := httpapi.New(options.HttpApiEndpoint, options)
runner.Logger.Info().Msgf("Listening api endpoint on: %s", options.HttpApiEndpoint)
gologger.Info().Msgf("Listening api endpoint on: %s", options.HttpApiEndpoint)
runner.httpApiEndpoint = apiServer
go func() {
if err := apiServer.Start(); err != nil {
runner.Logger.Error().Msgf("Failed to start API server: %s", err)
gologger.Error().Msgf("Failed to start API server: %s", err)
}
}()
}
@@ -246,23 +239,8 @@ func New(options *types.Options) (*Runner, error) {
os.Exit(0)
}
tmpDir, err := os.MkdirTemp("", "nuclei-tmp-*")
if err != nil {
return nil, errors.Wrap(err, "could not create temporary directory")
}
runner.tmpDir = tmpDir
// Cleanup tmpDir only if initialization fails
// On successful initialization, Close() method will handle cleanup
cleanupOnError := true
defer func() {
if cleanupOnError && runner.tmpDir != "" {
_ = os.RemoveAll(runner.tmpDir)
}
}()
// create the input provider and load the inputs
inputProvider, err := provider.NewInputProvider(provider.InputOptions{Options: options, TempDir: runner.tmpDir})
inputProvider, err := provider.NewInputProvider(provider.InputOptions{Options: options})
if err != nil {
return nil, errors.Wrap(err, "could not create input provider")
}
@@ -306,7 +284,7 @@ func New(options *types.Options) (*Runner, error) {
// create the resume configuration structure
resumeCfg := types.NewResumeCfg()
if runner.options.ShouldLoadResume() {
runner.Logger.Info().Msg("Resuming from save checkpoint")
gologger.Info().Msg("Resuming from save checkpoint")
file, err := os.ReadFile(runner.options.Resume)
if err != nil {
return nil, err
@@ -348,7 +326,6 @@ func New(options *types.Options) (*Runner, error) {
}
opts := interactsh.DefaultOptions(runner.output, runner.issuesClient, runner.progress)
opts.Logger = runner.Logger
opts.Debug = runner.options.Debug
opts.NoColor = runner.options.NoColor
if options.InteractshURL != "" {
@@ -378,28 +355,34 @@ func New(options *types.Options) (*Runner, error) {
}
interactshClient, err := interactsh.New(opts)
if err != nil {
runner.Logger.Error().Msgf("Could not create interactsh client: %s", err)
gologger.Error().Msgf("Could not create interactsh client: %s", err)
} else {
runner.interactsh = interactshClient
}
if options.RateLimitMinute > 0 {
runner.Logger.Print().Msgf("[%v] %v", aurora.BrightYellow("WRN"), "rate limit per minute is deprecated - use rate-limit-duration")
gologger.Print().Msgf("[%v] %v", aurora.BrightYellow("WRN"), "rate limit per minute is deprecated - use rate-limit-duration")
options.RateLimit = options.RateLimitMinute
options.RateLimitDuration = time.Minute
}
if options.RateLimit > 0 && options.RateLimitDuration == 0 {
options.RateLimitDuration = time.Second
}
runner.rateLimiter = utils.GetRateLimiter(context.Background(), options.RateLimit, options.RateLimitDuration)
if options.RateLimit == 0 && options.RateLimitDuration == 0 {
runner.rateLimiter = ratelimit.NewUnlimited(context.Background())
} else {
runner.rateLimiter = ratelimit.New(context.Background(), uint(options.RateLimit), options.RateLimitDuration)
}
if tmpDir, err := os.MkdirTemp("", "nuclei-tmp-*"); err == nil {
runner.tmpDir = tmpDir
}
// Initialization successful, disable cleanup on error
cleanupOnError = false
return runner, nil
}
// runStandardEnumeration runs standard enumeration
func (r *Runner) runStandardEnumeration(executerOpts *protocols.ExecutorOptions, store *loader.Store, engine *core.Engine) (*atomic.Bool, error) {
func (r *Runner) runStandardEnumeration(executerOpts protocols.ExecutorOptions, store *loader.Store, engine *core.Engine) (*atomic.Bool, error) {
if r.options.AutomaticScan {
return r.executeSmartWorkflowInput(executerOpts, store, engine)
}
@@ -430,7 +413,7 @@ func (r *Runner) Close() {
if r.inputProvider != nil {
r.inputProvider.Close()
}
protocolinit.Close(r.options.ExecutionId)
protocolinit.Close()
if r.pprofServer != nil {
r.pprofServer.Stop()
}
@@ -456,22 +439,23 @@ func (r *Runner) setupPDCPUpload(writer output.Writer) output.Writer {
if r.options.ScanID != "" {
r.options.EnableCloudUpload = true
}
if !r.options.EnableCloudUpload && !EnableCloudUpload {
r.pdcpUploadErrMsg = "Scan results upload to cloud is disabled."
if !(r.options.EnableCloudUpload || EnableCloudUpload) {
r.pdcpUploadErrMsg = fmt.Sprintf("[%v] Scan results upload to cloud is disabled.", r.colorizer.BrightYellow("WRN"))
return writer
}
color := aurora.NewAurora(!r.options.NoColor)
h := &pdcpauth.PDCPCredHandler{}
creds, err := h.GetCreds()
if err != nil {
if err != pdcpauth.ErrNoCreds && !HideAutoSaveMsg {
r.Logger.Verbose().Msgf("Could not get credentials for cloud upload: %s\n", err)
gologger.Verbose().Msgf("Could not get credentials for cloud upload: %s\n", err)
}
r.pdcpUploadErrMsg = fmt.Sprintf("To view results on Cloud Dashboard, configure API key from %v", pdcpauth.DashBoardURL)
r.pdcpUploadErrMsg = fmt.Sprintf("[%v] To view results on Cloud Dashboard, Configure API key from %v", color.BrightYellow("WRN"), pdcpauth.DashBoardURL)
return writer
}
uploadWriter, err := pdcp.NewUploadWriter(context.Background(), r.Logger, creds)
uploadWriter, err := pdcp.NewUploadWriter(context.Background(), creds)
if err != nil {
r.pdcpUploadErrMsg = fmt.Sprintf("PDCP (%v) Auto-Save Failed: %s\n", pdcpauth.DashBoardURL, err)
r.pdcpUploadErrMsg = fmt.Sprintf("[%v] PDCP (%v) Auto-Save Failed: %s\n", color.BrightYellow("WRN"), pdcpauth.DashBoardURL, err)
return writer
}
if r.options.ScanID != "" {
@@ -507,7 +491,6 @@ func (r *Runner) RunEnumeration() error {
Parser: r.parser,
TemporaryDirectory: r.tmpDir,
FuzzStatsDB: r.fuzzStats,
Logger: r.Logger,
}
dastServer, err := server.New(&server.Options{
Address: r.options.DASTServerAddress,
@@ -549,7 +532,7 @@ func (r *Runner) RunEnumeration() error {
// Create the executor options which will be used throughout the execution
// stage by the nuclei engine modules.
executorOpts := &protocols.ExecutorOptions{
executorOpts := protocols.ExecutorOptions{
Output: r.output,
Options: r.options,
Progress: r.progress,
@@ -567,8 +550,6 @@ func (r *Runner) RunEnumeration() error {
Parser: r.parser,
FuzzParamsFrequency: fuzzFreqCache,
GlobalMatchers: globalmatchers.New(),
DoNotCache: r.options.DoNotCacheTemplates,
Logger: r.Logger,
}
if config.DefaultConfig.IsDebugArgEnabled(config.DebugExportURLPattern) {
@@ -577,9 +558,7 @@ func (r *Runner) RunEnumeration() error {
}
if len(r.options.SecretsFile) > 0 && !r.options.Validate {
// Clone options so GetAuthTmplStore can modify them without affecting the original
authOptions := r.options.Copy()
authTmplStore, err := GetAuthTmplStore(authOptions, r.catalog, executorOpts)
authTmplStore, err := GetAuthTmplStore(*r.options, r.catalog, executorOpts)
if err != nil {
return errors.Wrap(err, "failed to load dynamic auth templates")
}
@@ -599,8 +578,8 @@ func (r *Runner) RunEnumeration() error {
if r.options.ShouldUseHostError() {
maxHostError := r.options.MaxHostError
if r.options.TemplateThreads > maxHostError {
r.Logger.Print().Msgf("[%v] The concurrency value is higher than max-host-error", r.colorizer.BrightYellow("WRN"))
r.Logger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", r.options.TemplateThreads)
gologger.Print().Msgf("[%v] The concurrency value is higher than max-host-error", r.colorizer.BrightYellow("WRN"))
gologger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", r.options.TemplateThreads)
maxHostError = r.options.TemplateThreads
}
@@ -615,7 +594,7 @@ func (r *Runner) RunEnumeration() error {
executorEngine := core.New(r.options)
executorEngine.SetExecuterOptions(executorOpts)
workflowLoader, err := parsers.NewLoader(executorOpts)
workflowLoader, err := parsers.NewLoader(&executorOpts)
if err != nil {
return errors.Wrap(err, "Could not create loader.")
}
@@ -654,7 +633,7 @@ func (r *Runner) RunEnumeration() error {
return err
}
if stats.GetValue(templates.SyntaxErrorStats) == 0 && stats.GetValue(templates.SyntaxWarningStats) == 0 && stats.GetValue(templates.RuntimeWarningsStats) == 0 {
r.Logger.Info().Msgf("All templates validated successfully")
gologger.Info().Msgf("All templates validated successfully\n")
} else {
return errors.New("encountered errors while performing template validation")
}
@@ -662,6 +641,7 @@ func (r *Runner) RunEnumeration() error {
}
store.Load()
// TODO: remove below functions after v3 or update warning messages
disk.PrintDeprecatedPathsMsgIfApplicable(r.options.Silent)
templates.PrintDeprecatedProtocolNameMsgIfApplicable(r.options.Silent, r.options.Verbose)
// add the hosts from the metadata queries of loaded templates into input provider
@@ -675,7 +655,7 @@ func (r *Runner) RunEnumeration() error {
}
ret := uncover.GetUncoverTargetsFromMetadata(context.TODO(), store.Templates(), r.options.UncoverField, uncoverOpts)
for host := range ret {
_ = r.inputProvider.SetWithExclusions(r.options.ExecutionId, host)
_ = r.inputProvider.SetWithExclusions(host)
}
}
// display execution info like version , templates used etc
@@ -683,7 +663,7 @@ func (r *Runner) RunEnumeration() error {
// prefetch secrets if enabled
if executorOpts.AuthProvider != nil && r.options.PreFetchSecrets {
r.Logger.Info().Msgf("Pre-fetching secrets from authprovider[s]")
gologger.Info().Msgf("Pre-fetching secrets from authprovider[s]")
if err := executorOpts.AuthProvider.PreFetchSecrets(); err != nil {
return errors.Wrap(err, "could not pre-fetch secrets")
}
@@ -717,7 +697,7 @@ func (r *Runner) RunEnumeration() error {
if r.dastServer != nil {
go func() {
if err := r.dastServer.Start(); err != nil {
r.Logger.Error().Msgf("could not start dast server: %v", err)
gologger.Error().Msgf("could not start dast server: %v", err)
}
}()
}
@@ -751,10 +731,10 @@ func (r *Runner) RunEnumeration() error {
// todo: error propagation without canonical straight error check is required by cloud?
// use safe dereferencing to avoid potential panics in case of previous unchecked errors
if v := ptrutil.Safe(results); !v.Load() {
r.Logger.Info().Msgf("Scan completed in %s. No results found.", shortDur(timeTaken))
gologger.Info().Msgf("Scan completed in %s. No results found.", shortDur(timeTaken))
} else {
matchCount := r.output.ResultCount()
r.Logger.Info().Msgf("Scan completed in %s. %d matches found.", shortDur(timeTaken), matchCount)
gologger.Info().Msgf("Scan completed in %s. %d matches found.", shortDur(timeTaken), matchCount)
}
// check if a passive scan was requested but no target was provided
@@ -795,7 +775,7 @@ func (r *Runner) isInputNonHTTP() bool {
return nonURLInput
}
func (r *Runner) executeSmartWorkflowInput(executorOpts *protocols.ExecutorOptions, store *loader.Store, engine *core.Engine) (*atomic.Bool, error) {
func (r *Runner) executeSmartWorkflowInput(executorOpts protocols.ExecutorOptions, store *loader.Store, engine *core.Engine) (*atomic.Bool, error) {
r.progress.Init(r.inputProvider.Count(), 0, 0)
service, err := automaticscan.New(automaticscan.Options{
@@ -863,7 +843,7 @@ func (r *Runner) displayExecutionInfo(store *loader.Store) {
if tmplCount == 0 && workflowCount == 0 {
// if dast flag is used print explicit warning
if r.options.DAST {
r.Logger.Print().Msgf("[%v] No DAST templates found", aurora.BrightYellow("WRN"))
gologger.DefaultLogger.Print().Msgf("[%v] No DAST templates found", aurora.BrightYellow("WRN"))
}
stats.ForceDisplayWarning(templates.SkippedCodeTmplTamperedStats)
} else {
@@ -883,38 +863,38 @@ func (r *Runner) displayExecutionInfo(store *loader.Store) {
return fmt.Sprintf("Current %s version: %v %v", versionType, version, updateutils.GetVersionDescription(version, latestVersion))
}
gologger.Info().Msg(versionInfo(config.Version, cfg.LatestNucleiVersion, "nuclei"))
gologger.Info().Msg(versionInfo(cfg.TemplateVersion, cfg.LatestNucleiTemplatesVersion, "nuclei-templates"))
gologger.Info().Msgf(versionInfo(config.Version, cfg.LatestNucleiVersion, "nuclei"))
gologger.Info().Msgf(versionInfo(cfg.TemplateVersion, cfg.LatestNucleiTemplatesVersion, "nuclei-templates"))
if !HideAutoSaveMsg {
if r.pdcpUploadErrMsg != "" {
r.Logger.Warning().Msgf("%s", r.pdcpUploadErrMsg)
gologger.Print().Msgf("%s", r.pdcpUploadErrMsg)
} else {
r.Logger.Info().Msgf("To view results on cloud dashboard, visit %v/scans upon scan completion.", pdcpauth.DashBoardURL)
gologger.Info().Msgf("To view results on cloud dashboard, visit %v/scans upon scan completion.", pdcpauth.DashBoardURL)
}
}
if tmplCount > 0 || workflowCount > 0 {
if len(store.Templates()) > 0 {
r.Logger.Info().Msgf("New templates added in latest release: %d", len(config.DefaultConfig.GetNewAdditions()))
r.Logger.Info().Msgf("Templates loaded for current scan: %d", len(store.Templates()))
gologger.Info().Msgf("New templates added in latest release: %d", len(config.DefaultConfig.GetNewAdditions()))
gologger.Info().Msgf("Templates loaded for current scan: %d", len(store.Templates()))
}
if len(store.Workflows()) > 0 {
r.Logger.Info().Msgf("Workflows loaded for current scan: %d", len(store.Workflows()))
gologger.Info().Msgf("Workflows loaded for current scan: %d", len(store.Workflows()))
}
for k, v := range templates.SignatureStats {
value := v.Load()
if value > 0 {
if k == templates.Unsigned && !r.options.Silent && !config.DefaultConfig.HideTemplateSigWarning {
r.Logger.Print().Msgf("[%v] Loading %d unsigned templates for scan. Use with caution.", r.colorizer.BrightYellow("WRN"), value)
gologger.Print().Msgf("[%v] Loading %d unsigned templates for scan. Use with caution.", r.colorizer.BrightYellow("WRN"), value)
} else {
r.Logger.Info().Msgf("Executing %d signed templates from %s", value, k)
gologger.Info().Msgf("Executing %d signed templates from %s", value, k)
}
}
}
}
if r.inputProvider.Count() > 0 {
r.Logger.Info().Msgf("Targets loaded for current scan: %d", r.inputProvider.Count())
gologger.Info().Msgf("Targets loaded for current scan: %d", r.inputProvider.Count())
}
}
@@ -941,7 +921,7 @@ func UploadResultsToCloud(options *types.Options) error {
return errors.Wrap(err, "could not get credentials for cloud upload")
}
ctx := context.TODO()
uploadWriter, err := pdcp.NewUploadWriter(ctx, options.Logger, creds)
uploadWriter, err := pdcp.NewUploadWriter(ctx, creds)
if err != nil {
return errors.Wrap(err, "could not create upload writer")
}
@@ -960,21 +940,19 @@ func UploadResultsToCloud(options *types.Options) error {
if err != nil {
return errors.Wrap(err, "could not open scan upload file")
}
defer func() {
_ = file.Close()
}()
defer file.Close()
options.Logger.Info().Msgf("Uploading scan results to cloud dashboard from %s", options.ScanUploadFile)
gologger.Info().Msgf("Uploading scan results to cloud dashboard from %s", options.ScanUploadFile)
dec := json.NewDecoder(file)
for dec.More() {
var r output.ResultEvent
err := dec.Decode(&r)
if err != nil {
options.Logger.Warning().Msgf("Could not decode jsonl: %s\n", err)
gologger.Warning().Msgf("Could not decode jsonl: %s\n", err)
continue
}
if err = uploadWriter.Write(&r); err != nil {
options.Logger.Warning().Msgf("[%s] failed to upload: %s\n", r.TemplateID, err)
gologger.Warning().Msgf("[%s] failed to upload: %s\n", r.TemplateID, err)
}
}
uploadWriter.Close()
+12 -12
View File
@@ -64,8 +64,8 @@ func TestWalkReflectStructAssignsEnvVars(t *testing.T) {
B: "$VAR_TWO",
},
}
_ = os.Setenv("VAR_EXAMPLE", "value")
_ = os.Setenv("VAR_TWO", "value2")
os.Setenv("VAR_EXAMPLE", "value")
os.Setenv("VAR_TWO", "value2")
Walk(testStruct, expandEndVars)
@@ -79,9 +79,9 @@ func TestWalkReflectStructHandlesDifferentTypes(t *testing.T) {
B: "$VAR_TWO",
C: "$VAR_THREE",
}
_ = os.Setenv("VAR_EXAMPLE", "value")
_ = os.Setenv("VAR_TWO", "2")
_ = os.Setenv("VAR_THREE", "true")
os.Setenv("VAR_EXAMPLE", "value")
os.Setenv("VAR_TWO", "2")
os.Setenv("VAR_THREE", "true")
Walk(testStruct, expandEndVars)
@@ -96,9 +96,9 @@ func TestWalkReflectStructEmpty(t *testing.T) {
B: "",
C: "$VAR_THREE",
}
_ = os.Setenv("VAR_EXAMPLE", "value")
_ = os.Setenv("VAR_TWO", "2")
_ = os.Setenv("VAR_THREE", "true")
os.Setenv("VAR_EXAMPLE", "value")
os.Setenv("VAR_TWO", "2")
os.Setenv("VAR_THREE", "true")
Walk(testStruct, expandEndVars)
@@ -116,7 +116,7 @@ func TestWalkReflectStructWithNoYamlTag(t *testing.T) {
C: "$GITHUB_USER",
}
_ = os.Setenv("GITHUB_USER", "testuser")
os.Setenv("GITHUB_USER", "testuser")
Walk(test, expandEndVars)
require.Equal(t, "testuser", test.A)
@@ -132,9 +132,9 @@ func TestWalkReflectStructHandlesNestedStructs(t *testing.T) {
C: "$VAR_THREE",
},
}
_ = os.Setenv("VAR_EXAMPLE", "value")
_ = os.Setenv("VAR_TWO", "2")
_ = os.Setenv("VAR_THREE", "true")
os.Setenv("VAR_EXAMPLE", "value")
os.Setenv("VAR_TWO", "2")
os.Setenv("VAR_THREE", "true")
Walk(testStruct, expandEndVars)
+12 -17
View File
@@ -12,6 +12,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
@@ -24,7 +25,7 @@ func (r *Runner) logAvailableTemplate(tplPath string) {
panic("not a template")
}
if err != nil {
r.Logger.Error().Msgf("Could not parse file '%s': %s\n", tplPath, err)
gologger.Error().Msgf("Could not parse file '%s': %s\n", tplPath, err)
} else {
r.verboseTemplate(tpl)
}
@@ -32,45 +33,39 @@ func (r *Runner) logAvailableTemplate(tplPath string) {
// log available templates for verbose (-vv)
func (r *Runner) verboseTemplate(tpl *templates.Template) {
r.Logger.Print().Msgf("%s\n", templates.TemplateLogMessage(tpl.ID,
gologger.Print().Msgf("%s\n", templates.TemplateLogMessage(tpl.ID,
types.ToString(tpl.Info.Name),
tpl.Info.Authors.ToSlice(),
tpl.Info.SeverityHolder.Severity))
}
func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
r.Logger.Print().Msgf(
gologger.Print().Msgf(
"\nListing available %v nuclei templates for %v",
config.DefaultConfig.TemplateVersion,
config.DefaultConfig.TemplatesDirectory,
)
// order templates alphabetically by path
templates := store.Templates()
sort.Slice(templates, func(i, j int) bool {
return templates[i].Path < templates[j].Path
})
for _, tpl := range templates {
for _, tpl := range store.Templates() {
if hasExtraFlags(r.options) {
if r.options.TemplateDisplay {
colorize := !r.options.NoColor
path := tpl.Path
tplBody, err := store.ReadTemplateFromURI(path, true)
if err != nil {
r.Logger.Error().Msgf("Could not read the template %s: %s", path, err)
gologger.Error().Msgf("Could not read the template %s: %s", path, err)
continue
}
if colorize {
path = aurora.Cyan(tpl.Path).String()
tplBody, err = r.highlightTemplate(&tplBody)
if err != nil {
r.Logger.Error().Msgf("Could not highlight the template %s: %s", tpl.Path, err)
gologger.Error().Msgf("Could not highlight the template %s: %s", tpl.Path, err)
continue
}
}
r.Logger.Print().Msgf("Template: %s\n\n%s", path, tplBody)
gologger.Silent().Msgf("Template: %s\n\n%s", path, tplBody)
} else {
r.Logger.Print().Msgf("%s\n", strings.TrimPrefix(tpl.Path, config.DefaultConfig.TemplatesDirectory+string(filepath.Separator)))
gologger.Silent().Msgf("%s\n", strings.TrimPrefix(tpl.Path, config.DefaultConfig.TemplatesDirectory+string(filepath.Separator)))
}
} else {
r.verboseTemplate(tpl)
@@ -79,7 +74,7 @@ func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
}
func (r *Runner) listAvailableStoreTags(store *loader.Store) {
r.Logger.Print().Msgf(
gologger.Print().Msgf(
"\nListing available %v nuclei tags for %v",
config.DefaultConfig.TemplateVersion,
config.DefaultConfig.TemplatesDirectory,
@@ -105,9 +100,9 @@ func (r *Runner) listAvailableStoreTags(store *loader.Store) {
for _, tag := range tagsList {
if r.options.JSONL {
marshalled, _ := jsoniter.Marshal(tag)
r.Logger.Debug().Msgf("%s", string(marshalled))
gologger.Silent().Msgf("%s\n", string(marshalled))
} else {
r.Logger.Debug().Msgf("%s (%d)", tag.Key, tag.Value)
gologger.Silent().Msgf("%s (%d)\n", tag.Key, tag.Value)
}
}
}
+4 -6
View File
@@ -41,7 +41,7 @@ type nucleiExecutor struct {
engine *core.Engine
store *loader.Store
options *NucleiExecutorOptions
executorOpts *protocols.ExecutorOptions
executorOpts protocols.ExecutorOptions
}
type NucleiExecutorOptions struct {
@@ -58,7 +58,6 @@ type NucleiExecutorOptions struct {
Colorizer aurora.Aurora
Parser parser.Parser
TemporaryDirectory string
Logger *gologger.Logger
}
func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
@@ -67,7 +66,7 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
// Create the executor options which will be used throughout the execution
// stage by the nuclei engine modules.
executorOpts := &protocols.ExecutorOptions{
executorOpts := protocols.ExecutorOptions{
Output: opts.Output,
Options: opts.Options,
Progress: opts.Progress,
@@ -86,7 +85,6 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
FuzzParamsFrequency: fuzzFreqCache,
GlobalMatchers: globalmatchers.New(),
FuzzStatsDB: opts.FuzzStatsDB,
Logger: opts.Logger,
}
if opts.Options.ShouldUseHostError() {
@@ -95,7 +93,7 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
maxHostError = 100 // auto adjust for fuzzings
}
if opts.Options.TemplateThreads > maxHostError {
opts.Logger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", opts.Options.TemplateThreads)
gologger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", opts.Options.TemplateThreads)
maxHostError = opts.Options.TemplateThreads
}
@@ -109,7 +107,7 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
executorEngine := core.New(opts.Options)
executorEngine.SetExecuterOptions(executorOpts)
workflowLoader, err := parsers.NewLoader(executorOpts)
workflowLoader, err := parsers.NewLoader(&executorOpts)
if err != nil {
return nil, errors.Wrap(err, "Could not create loader options.")
}
+2 -2
View File
@@ -112,7 +112,7 @@ func New(options *Options) (*DASTServer, error) {
func NewStatsServer(fuzzStatsDB *stats.Tracker) (*DASTServer, error) {
server := &DASTServer{
nucleiExecutor: &nucleiExecutor{
executorOpts: &protocols.ExecutorOptions{
executorOpts: protocols.ExecutorOptions{
FuzzStatsDB: fuzzStatsDB,
},
},
@@ -125,7 +125,7 @@ func NewStatsServer(fuzzStatsDB *stats.Tracker) (*DASTServer, error) {
func (s *DASTServer) Close() {
s.nucleiExecutor.Close()
_ = s.echo.Close()
s.echo.Close()
s.tasksPool.StopAndWaitFor(1 * time.Minute)
}
+1 -1
View File
@@ -37,7 +37,7 @@ import nuclei "github.com/projectdiscovery/nuclei/v3/lib"
## Advanced Example of using Nuclei Library/SDK
For Various use cases like batching etc. you might want to run nuclei in goroutines this can be done by using `nuclei.NewThreadSafeNucleiEngine`
For Various use cases like batching etc you might want to run nuclei in goroutines this can be done by using `nuclei.NewThreadSafeNucleiEngine`
```go
// create nuclei engine with options
+24 -73
View File
@@ -3,13 +3,11 @@ package nuclei
import (
"context"
"errors"
"os"
"time"
"github.com/projectdiscovery/goflags"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/utils/errkit"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog"
@@ -19,8 +17,8 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/hosterrorscache"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/vardump"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v3/pkg/templates/types"
pkgtypes "github.com/projectdiscovery/nuclei/v3/pkg/types"
)
// TemplateSources contains template sources
@@ -103,7 +101,7 @@ type InteractshOpts interactsh.Options
func WithInteractshOptions(opts InteractshOpts) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithInteractshOptions")
return ErrOptionsNotSupported.Msgf("WithInteractshOptions")
}
optsPtr := &opts
e.interactshOpts = (*interactsh.Options)(optsPtr)
@@ -181,7 +179,7 @@ func WithGlobalRateLimitCtx(ctx context.Context, maxTokens int, duration time.Du
return func(e *NucleiEngine) error {
e.opts.RateLimit = maxTokens
e.opts.RateLimitDuration = duration
e.rateLimiter = utils.GetRateLimiter(ctx, e.opts.RateLimit, e.opts.RateLimitDuration)
e.rateLimiter = ratelimit.New(ctx, uint(e.opts.RateLimit), e.opts.RateLimitDuration)
return nil
}
}
@@ -195,10 +193,8 @@ type HeadlessOpts struct {
}
// EnableHeadless allows execution of headless templates
//
// Warning: enabling headless mode may open up attack surface due to browser
// usage and can be prone to exploitation by custom unverified templates if not
// properly configured.
// *Use With Caution*: Enabling headless mode may open up attack surface due to browser usage
// and can be prone to exploitation by custom unverified templates if not properly configured
func EnableHeadlessWithOpts(hopts *HeadlessOpts) NucleiSDKOptions {
return func(e *NucleiEngine) error {
e.opts.Headless = true
@@ -208,6 +204,14 @@ func EnableHeadlessWithOpts(hopts *HeadlessOpts) NucleiSDKOptions {
e.opts.ShowBrowser = hopts.ShowBrowser
e.opts.UseInstalledChrome = hopts.UseChrome
}
if engine.MustDisableSandbox() {
gologger.Warning().Msgf("The current platform and privileged user will run the browser without sandbox\n")
}
browser, err := engine.New(e.opts)
if err != nil {
return err
}
e.browserInstance = browser
return nil
}
}
@@ -224,7 +228,7 @@ type StatsOptions struct {
func EnableStatsWithOpts(opts StatsOptions) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "EnableStatsWithOpts")
return ErrOptionsNotSupported.Msgf("EnableStatsWithOpts")
}
if opts.Interval == 0 {
opts.Interval = 5 //sec
@@ -252,7 +256,7 @@ type VerbosityOptions struct {
func WithVerbosity(opts VerbosityOptions) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithVerbosity")
return ErrOptionsNotSupported.Msgf("WithVerbosity")
}
e.opts.Verbose = opts.Verbose
e.opts.Silent = opts.Silent
@@ -285,15 +289,15 @@ type NetworkConfig struct {
func WithNetworkConfig(opts NetworkConfig) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithNetworkConfig")
return ErrOptionsNotSupported.Msgf("WithNetworkConfig")
}
e.opts.NoHostErrors = opts.DisableMaxHostErr
e.opts.MaxHostError = opts.MaxHostError
if e.opts.ShouldUseHostError() {
maxHostError := opts.MaxHostError
if e.opts.TemplateThreads > maxHostError {
e.Logger.Warning().Msg("The concurrency value is higher than max-host-error")
e.Logger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", e.opts.TemplateThreads)
gologger.Print().Msgf("[%v] The concurrency value is higher than max-host-error", e.executerOpts.Colorizer.BrightYellow("WRN"))
gologger.Info().Msgf("Adjusting max-host-error to the concurrency value: %d", e.opts.TemplateThreads)
maxHostError = e.opts.TemplateThreads
e.opts.MaxHostError = maxHostError
}
@@ -316,7 +320,7 @@ func WithNetworkConfig(opts NetworkConfig) NucleiSDKOptions {
func WithProxy(proxy []string, proxyInternalRequests bool) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithProxy")
return ErrOptionsNotSupported.Msgf("WithProxy")
}
e.opts.Proxy = proxy
e.opts.ProxyInternal = proxyInternalRequests
@@ -341,7 +345,7 @@ type OutputWriter output.Writer
func UseOutputWriter(writer OutputWriter) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "UseOutputWriter")
return ErrOptionsNotSupported.Msgf("UseOutputWriter")
}
e.customWriter = writer
return nil
@@ -356,7 +360,7 @@ type StatsWriter progress.Progress
func UseStatsWriter(writer StatsWriter) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "UseStatsWriter")
return ErrOptionsNotSupported.Msgf("UseStatsWriter")
}
e.customProgress = writer
return nil
@@ -370,7 +374,7 @@ func UseStatsWriter(writer StatsWriter) NucleiSDKOptions {
func WithTemplateUpdateCallback(disableTemplatesAutoUpgrade bool, callback func(newVersion string)) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithTemplateUpdateCallback")
return ErrOptionsNotSupported.Msgf("WithTemplateUpdateCallback")
}
e.disableTemplatesAutoUpgrade = disableTemplatesAutoUpgrade
e.onUpdateAvailableCallback = callback
@@ -382,7 +386,7 @@ func WithTemplateUpdateCallback(disableTemplatesAutoUpgrade bool, callback func(
func WithSandboxOptions(allowLocalFileAccess bool, restrictLocalNetworkAccess bool) NucleiSDKOptions {
return func(e *NucleiEngine) error {
if e.mode == threadSafe {
return errkit.Wrap(ErrOptionsNotSupported, "WithSandboxOptions")
return ErrOptionsNotSupported.Msgf("WithSandboxOptions")
}
e.opts.AllowLocalFileAccess = allowLocalFileAccess
e.opts.RestrictLocalNetworkAccess = restrictLocalNetworkAccess
@@ -415,14 +419,6 @@ func EnableGlobalMatchersTemplates() NucleiSDKOptions {
}
}
// DisableTemplateCache disables template caching
func DisableTemplateCache() NucleiSDKOptions {
return func(e *NucleiEngine) error {
e.opts.DoNotCacheTemplates = true
return nil
}
}
// EnableFileTemplates allows loading/executing file protocol templates
func EnableFileTemplates() NucleiSDKOptions {
return func(e *NucleiEngine) error {
@@ -467,14 +463,6 @@ func EnablePassiveMode() NucleiSDKOptions {
}
}
// EnableMatcherStatus allows enabling matcher status
func EnableMatcherStatus() NucleiSDKOptions {
return func(e *NucleiEngine) error {
e.opts.MatcherStatus = true
return nil
}
}
// WithAuthProvider allows setting a custom authprovider implementation
func WithAuthProvider(provider authprovider.AuthProvider) NucleiSDKOptions {
return func(e *NucleiEngine) error {
@@ -531,40 +519,3 @@ func WithResumeFile(file string) NucleiSDKOptions {
return nil
}
}
// WithLogger allows setting a shared gologger instance
func WithLogger(logger *gologger.Logger) NucleiSDKOptions {
return func(e *NucleiEngine) error {
e.Logger = logger
if e.opts != nil {
e.opts.Logger = logger
}
if e.executerOpts != nil {
e.executerOpts.Logger = logger
}
return nil
}
}
// WithOptions sets all options at once
func WithOptions(opts *pkgtypes.Options) NucleiSDKOptions {
return func(e *NucleiEngine) error {
e.opts = opts
return nil
}
}
// WithTemporaryDirectory allows setting a parent directory for SDK-managed temporary files.
// A temporary directory will be created inside the provided directory and cleaned up on engine close.
// If not set, a temporary directory will be automatically created in the system temp location.
// The parent directory is assumed to exist.
func WithTemporaryDirectory(parentDir string) NucleiSDKOptions {
return func(e *NucleiEngine) error {
tmpDir, err := os.MkdirTemp(parentDir, "nuclei-tmp-*")
if err != nil {
return err
}
e.tmpDir = tmpDir
return nil
}
}
+1
View File
@@ -1,4 +1,5 @@
//go:build !race
// +build !race
package nuclei_test
+19 -20
View File
@@ -12,9 +12,8 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/utils/errkit"
"github.com/rs/xid"
"github.com/projectdiscovery/ratelimit"
errorutil "github.com/projectdiscovery/utils/errors"
)
// unsafeOptions are those nuclei objects/instances/types
@@ -22,14 +21,14 @@ import (
// hence they are ephemeral and are created on every ExecuteNucleiWithOpts invocation
// in ThreadSafeNucleiEngine
type unsafeOptions struct {
executerOpts *protocols.ExecutorOptions
executerOpts protocols.ExecutorOptions
engine *core.Engine
}
// createEphemeralObjects creates ephemeral nuclei objects/instances/types
func createEphemeralObjects(ctx context.Context, base *NucleiEngine, opts *types.Options) (*unsafeOptions, error) {
u := &unsafeOptions{}
u.executerOpts = &protocols.ExecutorOptions{
u.executerOpts = protocols.ExecutorOptions{
Output: base.customWriter,
Options: opts,
Progress: base.customProgress,
@@ -53,7 +52,11 @@ func createEphemeralObjects(ctx context.Context, base *NucleiEngine, opts *types
if opts.RateLimit > 0 && opts.RateLimitDuration == 0 {
opts.RateLimitDuration = time.Second
}
u.executerOpts.RateLimiter = utils.GetRateLimiter(ctx, opts.RateLimit, opts.RateLimitDuration)
if opts.RateLimit == 0 && opts.RateLimitDuration == 0 {
u.executerOpts.RateLimiter = ratelimit.NewUnlimited(ctx)
} else {
u.executerOpts.RateLimiter = ratelimit.New(ctx, uint(opts.RateLimit), opts.RateLimitDuration)
}
u.engine = core.New(opts)
u.engine.SetExecuterOptions(u.executerOpts)
return u, nil
@@ -85,14 +88,10 @@ type ThreadSafeNucleiEngine struct {
// whose methods are thread-safe and can be used concurrently
// Note: Non-thread-safe methods start with Global prefix
func NewThreadSafeNucleiEngineCtx(ctx context.Context, opts ...NucleiSDKOptions) (*ThreadSafeNucleiEngine, error) {
defaultOptions := types.DefaultOptions()
defaultOptions.ExecutionId = xid.New().String()
// default options
e := &NucleiEngine{
opts: defaultOptions,
mode: threadSafe,
ctx: ctx,
Logger: defaultOptions.Logger,
opts: types.DefaultOptions(),
mode: threadSafe,
}
for _, option := range opts {
if err := option(e); err != nil {
@@ -122,12 +121,12 @@ func (e *ThreadSafeNucleiEngine) GlobalResultCallback(callback func(event *outpu
}
// ExecuteNucleiWithOptsCtx executes templates on targets and calls callback on each result(only if results are found)
// This method can be called concurrently and it will use some global resources but can be run parallelly
// This method can be called concurrently and it will use some global resources but can be runned parallelly
// by invoking this method with different options and targets
// Note: Not all options are thread-safe. this method will throw error if you try to use non-thread-safe options
func (e *ThreadSafeNucleiEngine) ExecuteNucleiWithOptsCtx(ctx context.Context, targets []string, opts ...NucleiSDKOptions) error {
baseOpts := e.eng.opts.Copy()
tmpEngine := &NucleiEngine{opts: baseOpts, mode: threadSafe}
baseOpts := *e.eng.opts
tmpEngine := &NucleiEngine{opts: &baseOpts, mode: threadSafe}
for _, option := range opts {
if err := option(tmpEngine); err != nil {
return err
@@ -143,19 +142,19 @@ func (e *ThreadSafeNucleiEngine) ExecuteNucleiWithOptsCtx(ctx context.Context, t
defer closeEphemeralObjects(unsafeOpts)
// load templates
workflowLoader, err := workflow.NewLoader(unsafeOpts.executerOpts)
workflowLoader, err := workflow.NewLoader(&unsafeOpts.executerOpts)
if err != nil {
return errkit.Wrapf(err, "Could not create workflow loader: %s", err)
return errorutil.New("Could not create workflow loader: %s\n", err)
}
unsafeOpts.executerOpts.WorkflowLoader = workflowLoader
store, err := loader.New(loader.NewConfig(tmpEngine.opts, e.eng.catalog, unsafeOpts.executerOpts))
if err != nil {
return errkit.Wrapf(err, "Could not create loader client: %s", err)
return errorutil.New("Could not create loader client: %s\n", err)
}
store.Load()
inputProvider := provider.NewSimpleInputProviderWithUrls(e.eng.opts.ExecutionId, targets...)
inputProvider := provider.NewSimpleInputProviderWithUrls(targets...)
if len(store.Templates()) == 0 && len(store.Workflows()) == 0 {
return ErrNoTemplatesAvailable
+23 -78
View File
@@ -5,10 +5,7 @@ import (
"bytes"
"context"
"io"
"os"
"sync"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
@@ -19,7 +16,6 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/progress"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/hosterrorscache"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolinit"
@@ -30,8 +26,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/ratelimit"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/projectdiscovery/utils/errkit"
"github.com/rs/xid"
errorutil "github.com/projectdiscovery/utils/errors"
)
// NucleiSDKOptions contains options for nuclei SDK
@@ -39,13 +34,13 @@ type NucleiSDKOptions func(e *NucleiEngine) error
var (
// ErrNotImplemented is returned when a feature is not implemented
ErrNotImplemented = errkit.New("Not implemented")
ErrNotImplemented = errorutil.New("Not implemented")
// ErrNoTemplatesAvailable is returned when no templates are available to execute
ErrNoTemplatesAvailable = errkit.New("No templates available")
ErrNoTemplatesAvailable = errorutil.New("No templates available")
// ErrNoTargetsAvailable is returned when no targets are available to scan
ErrNoTargetsAvailable = errkit.New("No targets available")
ErrNoTargetsAvailable = errorutil.New("No targets available")
// ErrOptionsNotSupported is returned when an option is not supported in thread safe mode
ErrOptionsNotSupported = errkit.New("Option not supported in thread safe mode")
ErrOptionsNotSupported = errorutil.NewWithFmt("Option %v not supported in thread safe mode")
)
type engineMode uint
@@ -69,7 +64,6 @@ type NucleiEngine struct {
templatesLoaded bool
// unexported core fields
ctx context.Context
interactshClient *interactsh.Client
catalog catalog.Catalog
rateLimiter *ratelimit.Limiter
@@ -90,26 +84,20 @@ type NucleiEngine struct {
customWriter output.Writer
customProgress progress.Progress
rc reporting.Client
executerOpts *protocols.ExecutorOptions
// Logger instance for the engine
Logger *gologger.Logger
// Temporary directory for SDK-managed template files
tmpDir string
executerOpts protocols.ExecutorOptions
}
// LoadAllTemplates loads all nuclei template based on given options
func (e *NucleiEngine) LoadAllTemplates() error {
workflowLoader, err := workflow.NewLoader(e.executerOpts)
workflowLoader, err := workflow.NewLoader(&e.executerOpts)
if err != nil {
return errkit.Wrapf(err, "Could not create workflow loader: %s", err)
return errorutil.New("Could not create workflow loader: %s\n", err)
}
e.executerOpts.WorkflowLoader = workflowLoader
e.store, err = loader.New(loader.NewConfig(e.opts, e.catalog, e.executerOpts))
if err != nil {
return errkit.Wrapf(err, "Could not create loader client: %s", err)
return errorutil.New("Could not create loader client: %s\n", err)
}
e.store.Load()
e.templatesLoaded = true
@@ -136,9 +124,9 @@ func (e *NucleiEngine) GetWorkflows() []*templates.Template {
func (e *NucleiEngine) LoadTargets(targets []string, probeNonHttp bool) {
for _, target := range targets {
if probeNonHttp {
_ = e.inputProvider.SetWithProbe(e.opts.ExecutionId, target, e.httpxClient)
_ = e.inputProvider.SetWithProbe(target, e.httpxClient)
} else {
e.inputProvider.Set(e.opts.ExecutionId, target)
e.inputProvider.Set(target)
}
}
}
@@ -148,9 +136,9 @@ func (e *NucleiEngine) LoadTargetsFromReader(reader io.Reader, probeNonHttp bool
buff := bufio.NewScanner(reader)
for buff.Scan() {
if probeNonHttp {
_ = e.inputProvider.SetWithProbe(e.opts.ExecutionId, buff.Text(), e.httpxClient)
_ = e.inputProvider.SetWithProbe(buff.Text(), e.httpxClient)
} else {
e.inputProvider.Set(e.opts.ExecutionId, buff.Text())
e.inputProvider.Set(buff.Text())
}
}
}
@@ -173,7 +161,7 @@ func (e *NucleiEngine) LoadTargetsWithHttpData(filePath string, filemode string)
// GetExecuterOptions returns the nuclei executor options
func (e *NucleiEngine) GetExecuterOptions() *protocols.ExecutorOptions {
return e.executerOpts
return &e.executerOpts
}
// ParseTemplate parses a template from given data
@@ -236,18 +224,12 @@ func (e *NucleiEngine) closeInternal() {
if e.httpxClient != nil {
_ = e.httpxClient.Close()
}
if e.tmpDir != "" {
_ = os.RemoveAll(e.tmpDir)
}
if e.opts != nil {
generators.ClearOptionsPayloadMap(e.opts)
}
}
// Close all resources used by nuclei engine
func (e *NucleiEngine) Close() {
e.closeInternal()
protocolinit.Close(e.opts.ExecutionId)
protocolinit.Close()
}
// ExecuteCallbackWithCtx executes templates on targets and calls callback on each result(only if results are found)
@@ -264,9 +246,9 @@ func (e *NucleiEngine) ExecuteCallbackWithCtx(ctx context.Context, callback ...f
}
filtered := []func(event *output.ResultEvent){}
for _, cb := range callback {
if cb != nil {
filtered = append(filtered, cb)
for _, callback := range callback {
if callback != nil {
filtered = append(filtered, callback)
}
}
e.resultCallbacks = append(e.resultCallbacks, filtered...)
@@ -276,32 +258,15 @@ func (e *NucleiEngine) ExecuteCallbackWithCtx(ctx context.Context, callback ...f
return ErrNoTemplatesAvailable
}
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
_ = e.engine.ExecuteScanWithOpts(ctx, templatesAndWorkflows, e.inputProvider, false)
}()
// wait for context to be cancelled
select {
case <-ctx.Done():
<-wait(&wg) // wait for scan to finish
return ctx.Err()
case <-wait(&wg):
// scan finished
}
_ = e.engine.ExecuteScanWithOpts(ctx, templatesAndWorkflows, e.inputProvider, false)
defer e.engine.WorkPool().Wait()
return nil
}
// ExecuteWithCallback is same as ExecuteCallbackWithCtx but with default context
// Note this is deprecated and will be removed in future major release
func (e *NucleiEngine) ExecuteWithCallback(callback ...func(event *output.ResultEvent)) error {
ctx := context.Background()
if e.ctx != nil {
ctx = e.ctx
}
return e.ExecuteCallbackWithCtx(ctx, callback...)
return e.ExecuteCallbackWithCtx(context.Background(), callback...)
}
// Options return nuclei Type Options
@@ -322,14 +287,9 @@ func (e *NucleiEngine) Store() *loader.Store {
// NewNucleiEngineCtx creates a new nuclei engine instance with given context
func NewNucleiEngineCtx(ctx context.Context, options ...NucleiSDKOptions) (*NucleiEngine, error) {
// default options
defaultOptions := types.DefaultOptions()
defaultOptions.ExecutionId = xid.New().String()
e := &NucleiEngine{
opts: defaultOptions,
mode: singleInstance,
ctx: ctx,
Logger: defaultOptions.Logger,
opts: types.DefaultOptions(),
mode: singleInstance,
}
for _, option := range options {
if err := option(e); err != nil {
@@ -346,18 +306,3 @@ func NewNucleiEngineCtx(ctx context.Context, options ...NucleiSDKOptions) (*Nucl
func NewNucleiEngine(options ...NucleiSDKOptions) (*NucleiEngine, error) {
return NewNucleiEngineCtx(context.Background(), options...)
}
// GetParser returns the template parser with cache
func (e *NucleiEngine) GetParser() *templates.Parser {
return e.parser
}
// wait for a waitgroup to finish
func wait(wg *sync.WaitGroup) <-chan struct{} {
ch := make(chan struct{})
go func() {
defer close(ch)
wg.Wait()
}()
return ch
}
+27 -76
View File
@@ -3,16 +3,15 @@ package nuclei
import (
"context"
"fmt"
"os"
"strings"
"sync"
"time"
"github.com/projectdiscovery/nuclei/v3/pkg/input"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/logrusorgru/aurora"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/httpx/common/httpx"
"github.com/projectdiscovery/nuclei/v3/internal/runner"
@@ -29,8 +28,8 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolinit"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -38,6 +37,8 @@ import (
"github.com/projectdiscovery/ratelimit"
)
var sharedInit *sync.Once
// applyRequiredDefaults to options
func (e *NucleiEngine) applyRequiredDefaults(ctx context.Context) {
mockoutput := testutils.NewMockOutputWriter(e.opts.OmitTemplate)
@@ -97,37 +98,27 @@ func (e *NucleiEngine) applyRequiredDefaults(ctx context.Context) {
// init
func (e *NucleiEngine) init(ctx context.Context) error {
// Update logger ref (if it was changed by [WithLogger])
// (Logger is already initialized)
if e.opts.Logger != e.Logger {
e.Logger = e.opts.Logger
}
if e.opts.Verbose {
e.Logger.SetMaxLevel(levels.LevelVerbose)
gologger.DefaultLogger.SetMaxLevel(levels.LevelVerbose)
} else if e.opts.Debug {
e.Logger.SetMaxLevel(levels.LevelDebug)
gologger.DefaultLogger.SetMaxLevel(levels.LevelDebug)
} else if e.opts.Silent {
e.Logger.SetMaxLevel(levels.LevelSilent)
gologger.DefaultLogger.SetMaxLevel(levels.LevelSilent)
}
if err := runner.ValidateOptions(e.opts); err != nil {
return err
}
if e.opts.Parser != nil {
if op, ok := e.opts.Parser.(*templates.Parser); ok {
e.parser = op
}
e.parser = templates.NewParser()
if sharedInit == nil || protocolstate.ShouldInit() {
sharedInit = &sync.Once{}
}
if e.parser == nil {
e.parser = templates.NewParser()
}
if protocolstate.ShouldInit(e.opts.ExecutionId) {
sharedInit.Do(func() {
_ = protocolinit.Init(e.opts)
}
})
if e.opts.ProxyInternal && e.opts.AliveHttpProxy != "" || e.opts.AliveSocksProxy != "" {
httpclient, err := httpclientpool.Get(e.opts, &httpclientpool.Configuration{})
@@ -165,50 +156,29 @@ func (e *NucleiEngine) init(ctx context.Context) error {
return err
}
if e.opts.Headless {
if engine.MustDisableSandbox() {
e.Logger.Warning().Msgf("The current platform and privileged user will run the browser without sandbox")
}
browser, err := engine.New(e.opts)
if err != nil {
return err
}
e.browserInstance = browser
}
if e.catalog == nil {
e.catalog = disk.NewCatalog(config.DefaultConfig.TemplatesDirectory)
}
if e.tmpDir == "" {
tmpDir, err := os.MkdirTemp("", "nuclei-tmp-*")
if err != nil {
return err
}
e.tmpDir = tmpDir
}
e.executerOpts = &protocols.ExecutorOptions{
Output: e.customWriter,
Options: e.opts,
Progress: e.customProgress,
Catalog: e.catalog,
IssuesClient: e.rc,
RateLimiter: e.rateLimiter,
Interactsh: e.interactshClient,
Colorizer: aurora.NewAurora(true),
ResumeCfg: types.NewResumeCfg(),
Browser: e.browserInstance,
Parser: e.parser,
InputHelper: input.NewHelper(),
TemporaryDirectory: e.tmpDir,
Logger: e.opts.Logger,
e.executerOpts = protocols.ExecutorOptions{
Output: e.customWriter,
Options: e.opts,
Progress: e.customProgress,
Catalog: e.catalog,
IssuesClient: e.rc,
RateLimiter: e.rateLimiter,
Interactsh: e.interactshClient,
Colorizer: aurora.NewAurora(true),
ResumeCfg: types.NewResumeCfg(),
Browser: e.browserInstance,
Parser: e.parser,
InputHelper: input.NewHelper(),
}
if e.opts.ShouldUseHostError() && e.hostErrCache != nil {
e.executerOpts.HostErrorsCache = e.hostErrCache
}
if len(e.opts.SecretsFile) > 0 {
authTmplStore, err := runner.GetAuthTmplStore(e.opts, e.catalog, e.executerOpts)
authTmplStore, err := runner.GetAuthTmplStore(*e.opts, e.catalog, e.executerOpts)
if err != nil {
return errors.Wrap(err, "failed to load dynamic auth templates")
}
@@ -250,25 +220,6 @@ func (e *NucleiEngine) init(ctx context.Context) error {
}
}
// Handle the case where the user passed an existing parser that we can use as a cache
if e.opts.Parser != nil {
if cachedParser, ok := e.opts.Parser.(*templates.Parser); ok {
e.parser = cachedParser
e.opts.Parser = cachedParser
e.executerOpts.Parser = cachedParser
e.executerOpts.Options.Parser = cachedParser
}
}
// Create a new parser if necessary
if e.parser == nil {
op := templates.NewParser()
e.parser = op
e.opts.Parser = op
e.executerOpts.Parser = op
e.executerOpts.Options.Parser = op
}
e.engine = core.New(e.opts)
e.engine.SetExecuterOptions(e.executerOpts)
Loaded 100 of 447 files, more files were not shown because too many files have changed in this diff. Show more