Compare commits

...
101 Commits
Author SHA1 Message Date
Sandeep Singh 10bf14a88a Merge pull request #766 from projectdiscovery/dev
v2.3.8 Bug fixes
2021-06-12 17:45:01 +05:30
sandeep 08bfdaface version update 2021-06-12 17:17:22 +05:30
Sandeep Singh d3b0a15284 Merge pull request #763 from projectdiscovery/fix-eof
Fix for panic: runtime error
2021-06-12 15:24:15 +05:30
Ice3man543 d22d823e9c Fixed #762 by not deleting file with no results 2021-06-12 03:48:10 +05:30
Sandeep Singh f6a680897c Merge pull request #731 from projectdiscovery/bugfix-709-http2
Updating retryablehttp with http2 support
2021-06-11 22:14:34 +05:30
Mzack9999 29e2ee7a0e Merge branch 'dev' into bugfix-709-http2 2021-06-11 17:20:40 +02:00
Ice3man543 b2e416b0dd Fixed nuclei panic for issue #760 2021-06-11 14:44:37 +05:30
Ice3man543 2ec27d4466 Fixed EOF read error 2021-06-09 11:15:21 +05:30
Sandeep Singh 673c98c3f4 Merge pull request #757 from projectdiscovery/sarif-integration
Added sarif output export format to nuclei
2021-06-08 16:47:46 +05:30
sandeep bd38c16e0a Update banner.go 2021-06-08 16:46:34 +05:30
sandeep 63b274ecb0 flag updates / improvements 2021-06-08 16:38:13 +05:30
Ice3man543 3ce48f3799 Misc 2021-06-06 17:38:39 +05:30
Ice3man543 8b52645905 Add name as rule help 2021-06-06 16:12:54 +05:30
Ice3man543 c913612fd3 Misc 2021-06-06 16:04:06 +05:30
Ice3man543 31a74a3390 Fix nil pointer panic 2021-06-06 15:57:22 +05:30
Ice3man543 7a962c8436 Correct offsets 2021-06-06 15:54:49 +05:30
Ice3man543 b99f028a66 Enrich file output sarif with correct file metadata 2021-06-06 15:52:13 +05:30
Ice3man543 2c0cea1a3f Changed name of flag 2021-06-06 02:56:36 +05:30
Ice3man543 5d744c7c20 Fixed reference printing bug 2021-06-06 01:04:20 +05:30
Ice3man543 eb96e701b5 Fixed reference formatting in reports 2021-06-05 23:43:37 +05:30
Ice3man543 e6db12848f Temporary fix for sarif github location issue 2021-06-05 23:42:37 +05:30
Ice3man543 1dc1b9ee73 Print first line of running action if any 2021-06-05 23:09:08 +05:30
Ice3man543 d9bb1393d3 Improvements to sarif report 2021-06-05 23:00:59 +05:30
Ice3man543 1977d41a1e Misc 2021-06-05 21:42:04 +05:30
Ice3man543 e09f16b68c Misc 2021-06-05 20:08:52 +05:30
Ice3man543 d3b2c8fee0 Improved output format 2021-06-05 20:06:23 +05:30
Ice3man543 abc7063339 Work on sarif integration start 2021-06-05 18:01:08 +05:30
Ice3man 84ebee3d4b Merge pull request #751 from gano3s/jira-on-prem
Integration with Jira Standalone server
2021-06-04 18:32:22 +05:30
ganoes 19157597a0 Added support of Jira on-prem server 2021-06-04 13:11:09 +02:00
Sandeep Singh 471f5c6b6b Merge pull request #747 from projectdiscovery/fix-master-crashes 2021-06-04 11:04:08 +05:30
Ice3man543 38abe40db8 Fixed #739 by adding normal request support to pipelining 2021-06-03 10:49:13 +05:30
Ice3man543 5a6303265d Fixed #745 crash with http clustering module
The crash is mitigated in two ways.
- An additional check in http clustering module for req condition (no clustering with conditions)
- Clustered Executer also now passes a non-nil map to mitigate the crash if for some reason we reach there.
2021-06-03 10:26:41 +05:30
Ice3man543 be776fb5dd Merge branch 'master' of https://github.com/projectdiscovery/nuclei into dev 2021-06-03 10:21:46 +05:30
Sandeep Singh 849c7a45fc Merge pull request #744 from projectdiscovery/docker_multi_platfrom
Testing docker multi platform images
2021-06-02 14:28:58 +05:30
sandeep 791b025309 Update dockerhub-push.yml 2021-06-02 14:16:58 +05:30
sandeep cb5688ba51 Testing docker multi platform images 2021-06-02 14:07:49 +05:30
Sandeep Singh 6f51e2a4d1 Merge pull request #733 from projectdiscovery/bugfix-732
Deprecating golint
2021-05-27 15:39:05 +05:30
Sandeep Singh ec939fdaaf Merge pull request #738 from projectdiscovery/security-information
Adding security information
2021-05-27 10:39:03 +05:30
Sandeep Singh 72ddf04321 Adding security information 2021-05-27 10:38:30 +05:30
Mzack9999 68dd65fd38 Deprecating golint 2021-05-26 00:07:59 +02:00
Mzack9999 871dbe7352 Updating retryablehttp with http2 support 2021-05-25 17:44:04 +02:00
sandeep cf41a3cfac Removing txt ext from default ignore list 2021-05-25 06:41:13 +05:30
Sandeep Singh 41f52492f9 Merge pull request #703 from projectdiscovery/dev
Improvements to interactsh integration
2021-05-09 15:23:33 +05:30
Sandeep Singh 137dc19735 Merge pull request #702 from projectdiscovery/nuclei-interact-improve
Improvements to interactsh integration + no fatal for interact
2021-05-09 15:18:03 +05:30
sandeep a3ad9999b0 Merge branch 'dev' into nuclei-interact-improve 2021-05-09 15:07:39 +05:30
Sandeep Singh 1dcb3c0aca Merge pull request #702 from projectdiscovery/nuclei-interact-improve
Improvements to interactsh integration + no fatal for interact
2021-05-09 02:38:01 +05:30
Ice3man543 6c4495c0d1 Fixed lint errors 2021-05-09 02:19:23 +05:30
Ice3man543 f85a421550 Improvements to interactsh integration + no fatal for interact 2021-05-09 01:37:22 +05:30
Sandeep Singh ccafe5ce68 Merge pull request #701 from projectdiscovery/dev
Interactsh bugfix update
2021-05-08 15:32:27 +05:30
sandeep ecae98b8a0 version update 2021-05-08 15:27:11 +05:30
sandeep b949aa2fa3 go mod update 2021-05-08 15:26:36 +05:30
Sandeep Singh c19438ce84 Merge pull request #698 from projectdiscovery/dev
Interactsh <> nuclei integration release
2021-05-04 18:36:35 +05:30
Sandeep Singh 31fc50d2a5 Merge pull request #697 from projectdiscovery/bugfix-host-header
Better host header handling in RFC compliant requests
2021-05-04 18:24:22 +05:30
Mzack9999 9012db5f79 Adding custom header preference over template definition 2021-05-04 14:36:04 +02:00
sandeep e3e766c641 version update 2021-05-04 17:02:33 +05:30
Ice3man543 f064104bde Fixed tests 2021-05-03 17:21:30 +05:30
Ice3man 849e526f33 Merge pull request #694 from projectdiscovery/interactsh-server-support
Interactsh <> nuclei integration
2021-05-03 17:05:43 +05:30
Ice3man543 8394e696ca Support interactsh payloads in baseURL 2021-05-03 14:31:44 +05:30
Ice3man543 6a4f3e1336 Reporting added to OOB interaction results 2021-05-03 14:08:09 +05:30
Mzack9999 79dd0549cd Better host header handling in RFC comliant requests 2021-05-02 20:10:49 +02:00
Ice3man543 9c3b0b69a5 Merge branch 'interactsh-server-support' of https://github.com/projectdiscovery/nuclei into interactsh-server-support 2021-05-01 18:28:38 +05:30
Ice3man543 563c7ab045 Fixed lint errors 2021-05-01 18:28:24 +05:30
Sandeep Singh 33f1e23e87 Merge branch 'dev' into interactsh-server-support 2021-05-01 12:05:27 +05:30
sandeep d3641bfe12 mod update 2021-05-01 12:02:28 +05:30
Sandeep Singh 34b9e1f22a Merge pull request #691 from projectdiscovery/dev
Minor bugfix release
2021-04-28 10:57:31 +05:30
sandeep 4e390dbccf Version update 2021-04-28 10:55:33 +05:30
Ice3man f30f524b36 Merge pull request #688 from seb-elttam/dev
Fix missing IP in the JSON when URLs contain hostname:port
2021-04-27 14:22:28 +05:30
Seb Macke 627e418287 Fix missing IP in the JSON when URLs contain hostname:port 2021-04-23 22:20:09 +10:00
Ice3man543 d9145b1c8a Fixed DSL interactsh matchers not working 2021-04-19 13:13:08 +05:30
Ice3man 2b602ebe50 Merge pull request #683 from projectdiscovery/bugfix-shared-fastdialer
adding shared fastdialer + missing cleanup
2021-04-19 01:02:23 +05:30
Ice3man543 5e0ae41c37 Fixed headless test cases failing 2021-04-19 01:00:59 +05:30
Ice3man543 4a9949d399 Fixed lint errors 2021-04-19 00:55:33 +05:30
Ice3man543 e6825d5960 Better marker identification + Misc 2021-04-18 17:53:59 +05:30
Ice3man543 959f8935e7 Changed - to _ 2021-04-18 17:44:21 +05:30
Ice3man543 993416d77c Misc 2021-04-18 17:29:01 +05:30
Ice3man543 e8747fc2da Made interactsh default, fixed matcher panic 2021-04-18 17:27:45 +05:30
Ice3man543 33bf306f28 Working interactsh integration 2021-04-18 16:10:10 +05:30
mzack 54540f3a22 adding shared fastdialer + missing cleanup 2021-04-18 11:57:43 +02:00
Ice3man543 4c594627a9 Started work on interact.sh support 2021-04-16 16:56:41 +05:30
PD-Team 27f15a9c93 Merge pull request #674 from projectdiscovery/master
dev even to master
2021-04-13 19:22:11 +05:30
Ice3man 851d1c2605 Merge pull request #670 from yabeow/master
Fix can't set user agent in templates
2021-04-13 19:17:47 +05:30
Gia. Bui Dai e229a3eccc fix typos 2021-04-13 13:28:29 +07:00
Gia. Bui Dai 7f82270ea7 Fix can't set user agent in templates 2021-04-13 13:27:36 +07:00
sandeep 781a71e3d5 FAQ update 2021-04-08 23:45:22 +05:30
PD-Team 157d6634d1 Merge pull request #660 from projectdiscovery/dependabot/github_actions/golangci/golangci-lint-action-v2.5.2
chore(deps): bump golangci/golangci-lint-action from v2 to v2.5.2
2021-04-05 21:55:17 +05:30
dependabot[bot] 954aec0907 chore(deps): bump golangci/golangci-lint-action from v2 to v2.5.2
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from v2 to v2.5.2.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v2...5c56cd6c9dc07901af25baab6f2b0d9f3b7c3018)

Signed-off-by: dependabot[bot] <[email protected]>
2021-04-05 07:45:55 +00:00
sandeep dbaa573b97 version update 2021-04-02 22:06:59 +05:30
Ice3man543 459fe31e0b Lint error fix 2021-04-02 21:32:56 +05:30
Ice3man543 8da933af53 Misc 2021-04-02 18:44:28 +05:30
Ice3man543 0d0e3f5523 Merge branch 'master' of https://github.com/projectdiscovery/nuclei 2021-04-02 18:41:07 +05:30
Ice3man543 6804bd79e8 New ignore functionality + error to warning 2021-04-02 18:40:58 +05:30
sandeep 5ea05b385a version update 2021-04-01 01:46:40 +05:30
Ice3man543 8233efe921 Fixed custom user agent issue in headless 2021-04-01 01:43:48 +05:30
Ice3man543 973e0be3e1 MIsc 2021-04-01 01:39:25 +05:30
Ice3man543 8d3163a842 Fixed a small bug 2021-04-01 01:37:40 +05:30
Ice3man543 64c8c33a4f Fixed some bugs + honor UA 2021-04-01 01:35:32 +05:30
Ice3man543 718e4505a6 Fixed nuclei ignore issues + made random agent default 2021-03-31 21:27:40 +05:30
Ice3man543 12b03f34bb Misc 2021-03-28 22:43:46 +05:30
PD-Team 76a14e91ed Merge pull request #640 from projectdiscovery/master
dev == master
2021-03-22 18:34:02 +05:30
Ice3man543 385b4558cb Merge branch 'master' of https://github.com/projectdiscovery/nuclei into dev 2021-03-19 17:01:53 +05:30
PD-Team 5cd493680e Dev == Master (#620) 2021-03-14 12:37:41 +05:30
58 changed files with 1090 additions and 258 deletions

No files matched your search

+2 -2
View File
@@ -13,10 +13,10 @@ jobs:
- name: Checkout code
uses: actions/checkout@v2
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v2
uses: golangci/golangci-lint-action
with:
# Required: the version of golangci-lint is required and must be specified without patch version: we always use the latest patch version.
version: v1.33
version: latest
args: --timeout 5m
working-directory: v2/
+10 -12
View File
@@ -1,19 +1,17 @@
# dockerhub-push pushes docker build to dockerhub automatically
# on the creation of a new release
name: Publish to Dockerhub on creation of a new release
on:
release:
types: [published]
name: ◎ Docker Push
on:
workflow_dispatch:
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: actions/checkout@v2
- name: Publish to Dockerhub Registry
#pre: echo ::save-state name=RELEASE_VERSION::$(echo ${GITHUB_REF:10})
uses: elgohr/Publish-Docker-Github-Action@master
uses: dawidd6/action-docker-publish-changed@v3
with:
name: projectdiscovery/nuclei
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
tags: "latest" #"latest,${{ env.STATE_RELEASE_VERSION }}"
username: ${{secrets.DOCKER_USERNAME}}
password: ${{secrets.DOCKER_PASSWORD}}
platforms: linux/amd64,linux/arm64,linux/arm
tag: latest
+2 -9
View File
@@ -66,7 +66,6 @@ linters:
- gocritic
- gofmt
- goimports
- golint
#- gomnd
- goprintffuncname
- gosimple
@@ -89,6 +88,7 @@ linters:
- unused
- varcheck
- whitespace
- revive
# don't enable:
# - depguard
@@ -105,11 +105,4 @@ linters:
# - nestif
# - prealloc
# - testpackage
# - wsl
# golangci.com configuration
# https://github.com/golangci/golangci/wiki/Configuration
service:
golangci-lint-version: 1.33.x # use the fixed version to not introduce new linters unexpectedly
prepare:
- echo "here I can run custom commands, but no preparation needed for this repo"
# - wsl
+2 -2
View File
@@ -22,8 +22,8 @@
<a href="#for-developers-and-organisations">For Developers</a> •
<a href="https://nuclei.projectdiscovery.io/nuclei/get-started/">Documentation</a> •
<a href="#credits">Credits</a> •
<a href="#license">License</a> •
<a href="https://discord.gg/KECAGdH">Join Discord</a>
<a href="https://nuclei.projectdiscovery.io/faq/nuclei/">FAQs</a> •
<a href="https://discord.gg/projectdiscovery">Join Discord</a>
</p>
---
+5
View File
@@ -0,0 +1,5 @@
# Security Policy
## Reporting a Vulnerability
DO NOT CREATE AN ISSUE to report a security problem. Instead, please send an email to security@projectdiscovery.io and we will acknowledge it within 3 working days.
+14
View File
@@ -0,0 +1,14 @@
# Go parameters
GOCMD=go
GOBUILD=$(GOCMD) build
GOMOD=$(GOCMD) mod
GOTEST=$(GOCMD) test
GOGET=$(GOCMD) get
all: build
build:
$(GOBUILD) -v -ldflags="-extldflags=-static" -o "nuclei" cmd/nuclei/main.go
test:
$(GOTEST) -v ./...
tidy:
$(GOMOD) tidy
+5 -3
View File
@@ -32,14 +32,16 @@
# issue-label: ""
# jira contains configuration options for jira issue tracker
#jira:
#jira:
# # Cloud is the boolean which tells if Jira instance is running in the cloud or on-prem version is used
# cloud: true
# # URL is the jira application url
# url: ""
# # account-id is the account-id of the jira user
# # account-id is the account-id of the jira user or username in case of on-prem Jira
# account-id: ""
# # email is the email of the user for jira instance
# email: ""
# # token is the token for jira instance.
# # token is the token for jira instance or password in case of on-prem Jira
# token: ""
# # project-name is the name of the project.
# project-name: ""
+9 -2
View File
@@ -25,6 +25,7 @@ func main() {
gologger.Fatal().Msgf("Could not create runner: %s\n", err)
}
nucleiRunner.RunEnumeration()
nucleiRunner.Close()
}
func readConfig() {
@@ -52,7 +53,6 @@ based on templates offering massive extensibility and ease of use.`)
set.BoolVarP(&options.NoColor, "no-color", "nc", false, "Disable colors in output")
set.IntVar(&options.Timeout, "timeout", 5, "Time to wait in seconds before timeout")
set.IntVar(&options.Retries, "retries", 1, "Number of times to retry a failed request")
set.BoolVarP(&options.RandomAgent, "random-agent", "ra", false, "Use randomly selected HTTP User-Agent header value")
set.StringSliceVarP(&options.CustomHeaders, "header", "H", []string{}, "Custom Header.")
set.BoolVar(&options.Debug, "debug", false, "Debugging request and responses")
set.BoolVar(&options.DebugRequests, "debug-req", false, "Debugging request")
@@ -85,7 +85,14 @@ based on templates offering massive extensibility and ease of use.`)
set.BoolVar(&options.SystemResolvers, "system-resolvers", false, "Use system dns resolving as error fallback")
set.IntVar(&options.PageTimeout, "page-timeout", 20, "Seconds to wait for each page in headless")
set.BoolVarP(&options.NewTemplates, "new-templates", "nt", false, "Only run newly added templates")
set.StringVarP(&options.DiskExportDirectory, "disk-export", "de", "", "Directory on disk to export reports in markdown to")
set.StringVarP(&options.DiskExportDirectory, "markdown-export", "me", "", "Directory to export results in markdown format")
set.StringVarP(&options.SarifExport, "sarif-export", "se", "", "File to export results in sarif format")
set.BoolVar(&options.NoInteractsh, "no-interactsh", false, "Do not use interactsh server for blind interaction polling")
set.StringVar(&options.InteractshURL, "interactsh-url", "https://interact.sh", "Self Hosted Interactsh Server URL")
set.IntVar(&options.InteractionsCacheSize, "interactions-cache-size", 5000, "Number of requests to keep in interactions cache")
set.IntVar(&options.InteractionsEviction, "interactions-eviction", 60, "Number of seconds to wait before evicting requests from cache")
set.IntVar(&options.InteractionsPollDuration, "interactions-poll-duration", 5, "Number of seconds before each interaction poll request")
set.IntVar(&options.InteractionsColldownPeriod, "interactions-cooldown-period", 5, "Extra time for interaction polling before exiting")
_ = set.Parse()
if cfgFile != "" {
+6 -5
View File
@@ -16,12 +16,14 @@ require (
github.com/hashicorp/go-retryablehttp v0.6.8 // indirect
github.com/json-iterator/go v1.1.10
github.com/julienschmidt/httprouter v1.3.0
github.com/karlseguin/ccache v2.0.3+incompatible
github.com/karrick/godirwalk v1.16.1
github.com/logrusorgru/aurora v2.0.3+incompatible
github.com/mattn/go-runewidth v0.0.10 // indirect
github.com/miekg/dns v1.1.38
github.com/mitchellh/go-ps v1.0.0
github.com/olekukonko/tablewriter v0.0.5
github.com/owenrumney/go-sarif v1.0.4
github.com/pkg/errors v0.9.1
github.com/projectdiscovery/clistats v0.0.8
github.com/projectdiscovery/collaborator v0.0.2
@@ -29,9 +31,10 @@ require (
github.com/projectdiscovery/goflags v0.0.4
github.com/projectdiscovery/gologger v1.1.4
github.com/projectdiscovery/hmap v0.0.1
github.com/projectdiscovery/rawhttp v0.0.6
github.com/projectdiscovery/interactsh v0.0.3
github.com/projectdiscovery/rawhttp v0.0.7
github.com/projectdiscovery/retryabledns v1.0.10
github.com/projectdiscovery/retryablehttp-go v1.0.1
github.com/projectdiscovery/retryablehttp-go v1.0.2-0.20210524224054-9fbe1f2b0727
github.com/remeh/sizedwaitgroup v1.0.0
github.com/rivo/uniseg v0.2.0 // indirect
github.com/rs/xid v1.2.1
@@ -47,11 +50,9 @@ require (
go.uber.org/multierr v1.6.0
go.uber.org/ratelimit v0.1.0
golang.org/x/crypto v0.0.0-20210218145215-b8e89b74b9df // indirect
golang.org/x/net v0.0.0-20210119194325-5f4716e94777
golang.org/x/net v0.0.0-20210521195947-fe42d452be8f
golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 // indirect
golang.org/x/sys v0.0.0-20210218155724-8ebf48af031b // indirect
golang.org/x/text v0.3.4 // indirect
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324 // indirect
google.golang.org/appengine v1.6.7 // indirect
gopkg.in/yaml.v2 v2.4.0
+48 -10
View File
@@ -40,6 +40,7 @@ github.com/Masterminds/semver v1.4.2/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF0
github.com/Masterminds/vcs v1.13.0/go.mod h1:N09YCmOQr6RLxC6UNHzuVwAdodYbbnycGHSmwVJjcKA=
github.com/andygrunwald/go-jira v1.13.0 h1:vvIImGgX32bHfoiyUwkNo+/YrPnRczNarvhLOncP6dE=
github.com/andygrunwald/go-jira v1.13.0/go.mod h1:jYi4kFDbRPZTJdJOVJO4mpMMIwdB+rcZwSO58DzPd2I=
github.com/apparentlymart/go-textseg/v13 v13.0.0/go.mod h1:ZK2fH7c4NqDTLtiYLvIkEghdlcqw7yxLeM89kiTRPUo=
github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ=
github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
@@ -61,6 +62,8 @@ github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumC
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
github.com/eggsampler/acme/v3 v3.2.1 h1:Lfsrg3M2zt00QRnizOFzdpSfsS9oDvPsGrodXS/w1KI=
github.com/eggsampler/acme/v3 v3.2.1/go.mod h1:/qh0rKC/Dh7Jj+p4So7DbWmFNzC4dpcpK53r226Fhuo=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
@@ -73,6 +76,7 @@ github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMo
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-redis/redis v6.15.5+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
github.com/go-rod/rod v0.91.1 h1:7xIlC/bXCXosZqZUl2x6GVB8tv4yMQ4W/ZVdGVa1qYI=
github.com/go-rod/rod v0.91.1/go.mod h1:/W4lcZiCALPD603MnJGIvhtywP3R6yRB9EDfFfsHiiI=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -133,6 +137,8 @@ github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hf
github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.2.0 h1:qJYtXnJRWmpe7m/3XlyhrsLrEURqHRM2kxzoxXqyUDs=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
@@ -148,12 +154,20 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ
github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/jasonlvhit/gocron v0.0.1 h1:qTt5qF3b3srDjeOIR4Le1LfeyvoYzJlYpqvG7tJX5YU=
github.com/jasonlvhit/gocron v0.0.1/go.mod h1:k9a3TV8VcU73XZxfVHCHWMWF9SOqgoku0/QlY2yvlA4=
github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/karlseguin/ccache v2.0.3+incompatible h1:j68C9tWOROiOLWTS/kCGg9IcJG+ACqn5+0+t8Oh83UU=
github.com/karlseguin/ccache v2.0.3+incompatible/go.mod h1:CM9tNPzT6EdRh14+jiW8mEF9mkNZuuE51qmgGYUB93w=
github.com/karlseguin/ccache/v2 v2.0.7 h1:y5Pfi4eiyYCOD6LS/Kj+o6Nb4M5Ngpw9qFQs+v44ZYM=
github.com/karlseguin/ccache/v2 v2.0.7/go.mod h1:2BDThcfQMf/c0jnZowt16eW405XIqZPavt+HoYEtcxQ=
github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003 h1:vJ0Snvo+SLMY72r5J4sEfkuE7AFbixEP2qRbEcum/wA=
github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003/go.mod h1:zNBxMY8P21owkeogJELCLeHIt+voOSduHYTFUbwRAV8=
github.com/karrick/godirwalk v1.16.1 h1:DynhcF+bztK8gooS0+NDJFrdNZjJ3gzVzC545UNA9iw=
github.com/karrick/godirwalk v1.16.1/go.mod h1:j4mkqPuvaLI8mp1DroR3P6ad7cyYd4c1qeJ3RV7ULlk=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
@@ -164,6 +178,7 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381/go.mod h1:7rIyQOR62GCctdiQpZ/zOJlFyk6y+94wXzv6RNZgaR4=
github.com/logrusorgru/aurora v2.0.3+incompatible h1:tOpm7WcpBTn4fjmVfgpQq0EfczGlG91VSDkswnjF5A8=
github.com/logrusorgru/aurora v2.0.3+incompatible/go.mod h1:7rIyQOR62GCctdiQpZ/zOJlFyk6y+94wXzv6RNZgaR4=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
@@ -186,10 +201,14 @@ github.com/ngdinhtoan/glide-cleanup v0.2.0/go.mod h1:UQzsmiDOb8YV3nOsCxK/c9zPpCZ
github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0 h1:WSHQ+IS43OoUrWtD1/bbclrwK8TTH5hzp+umCiuxHgs=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.4.3 h1:RE1xgDvH7imwFD45h+u2SgIfERHlS2yNG4DObb5BSKU=
github.com/onsi/ginkgo v1.10.1 h1:q/mM8GF/n0shIN8SaAZ0V+jnLPzen6WIVZdiwrRlMlo=
github.com/onsi/ginkgo v1.10.1/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.0 h1:XPnZz8VVBHjVsy1vzJmRwIcSwiUO+JFfrv/xGiigmME=
github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/owenrumney/go-sarif v1.0.4 h1:0LFC5eHP6amc/9ajM1jDiE52UfXFcl/oozay+X3KgV4=
github.com/owenrumney/go-sarif v1.0.4/go.mod h1:DXUGbHwQcCMvqcvZbxh8l/7diHsJVztOKZgmPt88RNI=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -203,17 +222,23 @@ github.com/projectdiscovery/fastdialer v0.0.8 h1:mEMc8bfXV5hc1PUEkJiUnR5imYQe6+8
github.com/projectdiscovery/fastdialer v0.0.8/go.mod h1:AuaV0dzrNeBLHqjNnzpFSnTXnHGIZAlGQE+WUMmSIW4=
github.com/projectdiscovery/goflags v0.0.4 h1:fWKLMAr3KmPlZxE1b54pfei+vGIUJn9q6aM7woZIbCY=
github.com/projectdiscovery/goflags v0.0.4/go.mod h1:Ae1mJ5MIIqjys0lFe3GiMZ10Z8VLaxkYJ1ySA4Zv8HA=
github.com/projectdiscovery/gologger v1.1.3/go.mod h1:jdXflz3TLB8bcVNzb0v26TztI9KPz8Lr4BVdUhNUs6E=
github.com/projectdiscovery/gologger v1.1.4 h1:qWxGUq7ukHWT849uGPkagPKF3yBPYAsTtMKunQ8O2VI=
github.com/projectdiscovery/gologger v1.1.4/go.mod h1:Bhb6Bdx2PV1nMaFLoXNBmHIU85iROS9y1tBuv7T5pMY=
github.com/projectdiscovery/hmap v0.0.1 h1:VAONbJw5jP+syI5smhsfkrq9XPGn4aiYy5pR6KR1wog=
github.com/projectdiscovery/hmap v0.0.1/go.mod h1:VDEfgzkKQdq7iGTKz8Ooul0NuYHQ8qiDs6r8bPD1Sb0=
github.com/projectdiscovery/rawhttp v0.0.6 h1:HbgPB1eKXQVV5F9sq0Uxflm95spWFyZYD8dgFpeOC9M=
github.com/projectdiscovery/rawhttp v0.0.6/go.mod h1:PQERZAhAv7yxI/hR6hdDPgK1WTU56l204BweXrBec+0=
github.com/projectdiscovery/interactsh v0.0.3 h1:PUkWk+NzSyd5glMqfORmuqizhsd7c3WdTYBOto/MQIU=
github.com/projectdiscovery/interactsh v0.0.3/go.mod h1:dWnKO14d2FLP3kLhI9DecEsiAC/aZiJoUBGFjGhDskY=
github.com/projectdiscovery/rawhttp v0.0.7 h1:5m4peVgjbl7gqDcRYMTVEuX+Xs/nh76ohTkkvufucLg=
github.com/projectdiscovery/rawhttp v0.0.7/go.mod h1:PQERZAhAv7yxI/hR6hdDPgK1WTU56l204BweXrBec+0=
github.com/projectdiscovery/retryabledns v1.0.7/go.mod h1:/UzJn4I+cPdQl6pKiiQfvVAT636YZvJQYZhYhGB0dUQ=
github.com/projectdiscovery/retryabledns v1.0.10 h1:xJZ2aKoqrNg/OZEw1+4+QIOH40V/WkZDYY1ZZc+uphE=
github.com/projectdiscovery/retryabledns v1.0.10/go.mod h1:4sMC8HZyF01HXukRleSQYwz4870bwgb4+hTSXTMrkf4=
github.com/projectdiscovery/retryablehttp-go v1.0.1 h1:V7wUvsZNq1Rcz7+IlcyoyQlNwshuwptuBVYWw9lx8RE=
github.com/projectdiscovery/retryablehttp-go v1.0.1/go.mod h1:SrN6iLZilNG1X4neq1D+SBxoqfAF4nyzvmevkTkWsek=
github.com/projectdiscovery/retryablehttp-go v1.0.2-0.20210524224054-9fbe1f2b0727 h1:CJHP3CLCc/eqdXQEvZy8KiiqtAk9kEsd1URtPyPAQ1s=
github.com/projectdiscovery/retryablehttp-go v1.0.2-0.20210524224054-9fbe1f2b0727/go.mod h1:dx//aY9V247qHdsRf0vdWHTBZuBQ2vm6Dq5dagxrDYI=
github.com/prologic/smtpd v0.0.0-20210126001904-0893ad18168e h1:ZT3wZ92sp/EHEE/HcFCWCsYS3ROLjHb6EqSX8qYrgXw=
github.com/prologic/smtpd v0.0.0-20210126001904-0893ad18168e/go.mod h1:GkLsdH1RZj6RDKeI9A05NGZYmEZQ/PbQcZPnZoSZuYI=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/remeh/sizedwaitgroup v1.0.0 h1:VNGGFwNo/R5+MJBf6yrsr110p0m4/OX4S3DCy7Kyl5E=
github.com/remeh/sizedwaitgroup v1.0.0/go.mod h1:3j2R4OIe/SeS6YDhICBy22RWjJC5eNCJ1V+9+NVNYlo=
@@ -233,6 +258,7 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
@@ -244,6 +270,10 @@ github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6Kllzaw
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasttemplate v1.2.1 h1:TVEnxayobAdVkhQfrfes2IzOB6o+z4roRkPF52WA1u4=
github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/vmihailenco/msgpack/v4 v4.3.12/go.mod h1:gborTTJjAo/GWTqqRjrLCn9pgNN+NXzzngzBKDPIqw4=
github.com/vmihailenco/tagparser v0.1.1/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgqMEUPoW2WPyhdI=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0 h1:3UeQBvD0TFrlVjOeLOBz+CPAI8dnbqNSVwUwRrkp7vQ=
github.com/wsxiaoys/terminal v0.0.0-20160513160801-0940f3fc43a0/go.mod h1:IXCdmsXIht47RaVFLEdVnh1t+pgYtTAhQGj73kz+2DM=
github.com/xanzy/go-gitlab v0.44.0 h1:cEiGhqu7EpFGuei2a2etAwB+x6403E5CvpLn35y+GPs=
github.com/xanzy/go-gitlab v0.44.0/go.mod h1:sPLojNBn68fMUWSxIJtdVVIP8uSBYqesTfDUseX11Ug=
github.com/ysmood/goob v0.3.0 h1:XZ51cZJ4W3WCoCiUktixzMIQF86W7G5VFL4QQ/Q2uS0=
@@ -259,6 +289,8 @@ github.com/ysmood/leakless v0.6.12/go.mod h1:R8iAXPRaG97QJwqxs74RdwzcRHT1SWCGTNq
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/zclconf/go-cty v1.8.2 h1:u+xZfBKgpycDnTNjPhGiTEYZS5qS/Sb5MqSfm7vzcjg=
github.com/zclconf/go-cty v1.8.2/go.mod h1:vVKLxnk3puL4qRAv72AO+W99LUD4da90g3uUAzyuvAk=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
@@ -276,6 +308,7 @@ golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201112155050-0c6587e931a9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210218145215-b8e89b74b9df h1:y7QZzfUiTwWam+xBn29Ulb8CBwVN5UdzmMDavl9Whlw=
@@ -339,8 +372,9 @@ golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777 h1:003p0dJM77cxMSyCPFphvZf/Y5/NXf5fzg6ufd1/Oew=
golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210521195947-fe42d452be8f h1:Si4U+UcgJzya9kpiEUJKQvjr512OLli+gL4poHrz93U=
golang.org/x/net v0.0.0-20210521195947-fe42d452be8f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -387,11 +421,12 @@ golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201113233024-12cec1faf1ba/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210218155724-8ebf48af031b h1:lAZ0/chPUDWwjqosYR0X4M490zQhMsiJ4K3DbA7o+3g=
golang.org/x/sys v0.0.0-20210218155724-8ebf48af031b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -399,8 +434,9 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -533,6 +569,8 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/corvus-ch/zbase32.v1 v1.0.0 h1:K4u1NprbDNvKPczKfHLbwdOWHTZ0zfv2ow71H1nRnFU=
gopkg.in/corvus-ch/zbase32.v1 v1.0.0/go.mod h1:T3oKkPOm4AV/bNXCNFUxRmlE9RUyBz/DSo0nK9U+c0Y=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
+2 -2
View File
@@ -7,11 +7,11 @@ const banner = `
____ __ _______/ /__ (_)
/ __ \/ / / / ___/ / _ \/ /
/ / / / /_/ / /__/ / __/ /
/_/ /_/\__,_/\___/_/\___/_/ v2.3.2
/_/ /_/\__,_/\___/_/\___/_/ v2.3.8
`
// Version is the current version of nuclei
const Version = `2.3.2`
const Version = `2.3.8`
// showBanner is used to show the banner to the user
func showBanner() {
+15 -18
View File
@@ -1,14 +1,14 @@
package runner
import (
"bufio"
"os"
"path"
"regexp"
"strings"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/projectdiscovery/gologger"
"gopkg.in/yaml.v2"
)
// nucleiConfig contains some configuration options for nuclei
@@ -83,25 +83,27 @@ func (r *Runner) writeConfiguration(config *nucleiConfig) error {
const nucleiIgnoreFile = ".nuclei-ignore"
type ignoreFile struct {
Tags []string `yaml:"tags"`
Files []string `yaml:"files"`
}
// readNucleiIgnoreFile reads the nuclei ignore file marking it in map
func (r *Runner) readNucleiIgnoreFile() {
file, err := os.Open(r.getIgnoreFilePath())
if err != nil {
gologger.Error().Msgf("Could not read nuclei-ignore file: %s\n", err)
return
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
text := scanner.Text()
if text == "" {
continue
}
if strings.HasPrefix(text, "#") {
continue
}
r.templatesConfig.IgnorePaths = append(r.templatesConfig.IgnorePaths, text)
ignore := &ignoreFile{}
if err := yaml.NewDecoder(file).Decode(ignore); err != nil {
gologger.Error().Msgf("Could not parse nuclei-ignore file: %s\n", err)
return
}
r.options.ExcludeTags = append(r.options.ExcludeTags, ignore.Tags...)
r.templatesConfig.IgnorePaths = append(r.templatesConfig.IgnorePaths, ignore.Files...)
}
// getIgnoreFilePath returns the ignore file path for the runner
@@ -114,17 +116,12 @@ func (r *Runner) getIgnoreFilePath() string {
_ = os.MkdirAll(configDir, os.ModePerm)
defIgnoreFilePath = path.Join(configDir, nucleiIgnoreFile)
return defIgnoreFilePath
}
cwd, err := os.Getwd()
if err != nil {
return defIgnoreFilePath
}
cwdIgnoreFilePath := path.Join(cwd, nucleiIgnoreFile)
cwdIfpInfo, err := os.Stat(cwdIgnoreFilePath)
if os.IsNotExist(err) || cwdIfpInfo.IsDir() {
return defIgnoreFilePath
}
return cwdIgnoreFilePath
}
-1
View File
@@ -25,7 +25,6 @@ func ParseOptions(options *types.Options) {
// Show the user the banner
showBanner()
options.ExcludeTags = append(options.ExcludeTags, "dos")
if options.Version {
gologger.Info().Msgf("Current Version: %s\n", Version)
os.Exit(0)
+46 -10
View File
@@ -6,6 +6,7 @@ import (
"os"
"path"
"strings"
"time"
"github.com/logrusorgru/aurora"
"github.com/projectdiscovery/gologger"
@@ -18,9 +19,12 @@ import (
"github.com/projectdiscovery/nuclei/v2/pkg/projectfile"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/clusterer"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolinit"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/exporters/disk"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/exporters/sarif"
"github.com/projectdiscovery/nuclei/v2/pkg/templates"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
"github.com/remeh/sizedwaitgroup"
@@ -34,6 +38,7 @@ import (
type Runner struct {
hostMap *hybrid.HybridMap
output output.Writer
interactsh *interactsh.Client
inputCount int64
templatesConfig *nucleiConfig
options *types.Options
@@ -62,12 +67,13 @@ func New(options *types.Options) (*Runner, error) {
if err := runner.updateTemplates(); err != nil {
gologger.Warning().Msgf("Could not update templates: %s\n", err)
}
runner.catalog = catalog.New(runner.options.TemplatesDirectory)
// Read nucleiignore file if given a templateconfig
if runner.templatesConfig != nil {
runner.readNucleiIgnoreFile()
runner.catalog.AppendIgnore(runner.templatesConfig.IgnorePaths)
}
runner.catalog = catalog.New(runner.options.TemplatesDirectory)
var reportingOptions *reporting.Options
if options.ReportingConfig != "" {
file, err := os.Open(options.ReportingConfig)
@@ -90,6 +96,14 @@ func New(options *types.Options) (*Runner, error) {
reportingOptions.DiskExporter = &disk.Options{Directory: options.DiskExportDirectory}
}
}
if options.SarifExport != "" {
if reportingOptions != nil {
reportingOptions.SarifExporter = &sarif.Options{File: options.SarifExport}
} else {
reportingOptions = &reporting.Options{}
reportingOptions.SarifExporter = &sarif.Options{File: options.SarifExport}
}
}
if reportingOptions != nil {
if client, err := reporting.New(reportingOptions, options.ReportingDB); err != nil {
gologger.Fatal().Msgf("Could not create issue reporting client: %s\n", err)
@@ -195,6 +209,24 @@ func New(options *types.Options) (*Runner, error) {
}
}
if !options.NoInteractsh {
interactshClient, err := interactsh.New(&interactsh.Options{
ServerURL: options.InteractshURL,
CacheSize: int64(options.InteractionsCacheSize),
Eviction: time.Duration(options.InteractionsEviction) * time.Second,
ColldownPeriod: time.Duration(options.InteractionsColldownPeriod) * time.Second,
PollDuration: time.Duration(options.InteractionsPollDuration) * time.Second,
Output: runner.output,
IssuesClient: runner.issuesClient,
Progress: runner.progress,
})
if err != nil {
gologger.Error().Msgf("Could not create interactsh client: %s", err)
} else {
runner.interactsh = interactshClient
}
}
// Enable Polling
if options.BurpCollaboratorBiid != "" {
collaborator.DefaultCollaborator.Collab.AddBIID(options.BurpCollaboratorBiid)
@@ -217,6 +249,7 @@ func (r *Runner) Close() {
if r.projectFile != nil {
r.projectFile.Close()
}
protocolinit.Close()
}
// RunEnumeration sets up the input layer for giving input nuclei.
@@ -235,8 +268,8 @@ func (r *Runner) RunEnumeration() {
}
r.options.Templates = append(r.options.Templates, templatesLoaded...)
}
includedTemplates := r.catalog.GetTemplatesPath(r.options.Templates)
excludedTemplates := r.catalog.GetTemplatesPath(r.options.ExcludedTemplates)
includedTemplates := r.catalog.GetTemplatesPath(r.options.Templates, false)
excludedTemplates := r.catalog.GetTemplatesPath(r.options.ExcludedTemplates, true)
// defaults to all templates
allTemplates := includedTemplates
@@ -260,7 +293,7 @@ func (r *Runner) RunEnumeration() {
// pre-parse all the templates, apply filters
finalTemplates := []*templates.Template{}
workflowPaths := r.catalog.GetTemplatesPath(r.options.Workflows)
workflowPaths := r.catalog.GetTemplatesPath(r.options.Workflows, false)
availableTemplates, _ := r.getParsedTemplatesFor(allTemplates, r.options.Severity, false)
availableWorkflows, workflowCount := r.getParsedTemplatesFor(workflowPaths, r.options.Severity, true)
@@ -288,6 +321,7 @@ func (r *Runner) RunEnumeration() {
IssuesClient: r.issuesClient,
Browser: r.browser,
ProjectFile: r.projectFile,
Interactsh: r.interactsh,
}
clusterID := fmt.Sprintf("cluster-%s", xid.New().String())
@@ -349,19 +383,21 @@ func (r *Runner) RunEnumeration() {
}(t)
}
wgtemplates.Wait()
if r.interactsh != nil {
matched := r.interactsh.Close()
if matched {
results.CAS(false, true)
}
}
r.progress.Stop()
if r.issuesClient != nil {
r.issuesClient.Close()
}
if !results.Load() {
if r.output != nil {
r.output.Close()
os.Remove(r.options.Output)
}
gologger.Info().Msgf("No results found. Better luck next time!")
}
if r.browser != nil {
r.browser.Close()
}
+1
View File
@@ -62,6 +62,7 @@ func (r *Runner) parseTemplateFile(file string) (*templates.Template, error) {
Catalog: r.catalog,
IssuesClient: r.issuesClient,
RateLimiter: r.ratelimiter,
Interactsh: r.interactsh,
ProjectFile: r.projectFile,
Browser: r.browser,
}
+3 -2
View File
@@ -44,7 +44,7 @@ func (r *Runner) updateTemplates() error {
configDir := path.Join(home, "/.config", "/nuclei")
_ = os.MkdirAll(configDir, os.ModePerm)
templatesConfigFile := path.Join(home, nucleiConfigFilename)
templatesConfigFile := path.Join(configDir, nucleiConfigFilename)
if _, statErr := os.Stat(templatesConfigFile); !os.IsNotExist(statErr) {
config, readErr := readConfiguration()
if err != nil {
@@ -65,6 +65,7 @@ func (r *Runner) updateTemplates() error {
}
r.templatesConfig = currentConfig
}
// Check if last checked for nuclei-ignore is more than 1 hours.
// and if true, run the check.
if r.templatesConfig == nil || time.Since(r.templatesConfig.LastCheckedIgnore) > 1*time.Hour || r.options.UpdateTemplates {
@@ -316,7 +317,7 @@ func (r *Runner) compareAndWriteTemplates(z *zip.Reader) (*templateUpdateResults
paths := strings.Split(directory, "/")
finalPath := strings.Join(paths[1:], "/")
if (!strings.EqualFold(name, ".nuclei-ignore") && strings.HasPrefix(name, ".")) || strings.HasPrefix(finalPath, ".") || strings.EqualFold(name, "README.md") {
if strings.HasPrefix(name, ".") || strings.HasPrefix(finalPath, ".") || strings.EqualFold(name, "README.md") {
continue
}
results.totalCount++
-1
View File
@@ -19,7 +19,6 @@ func Init(options *types.Options) {
// DefaultOptions is the default options structure for nuclei during mocking.
var DefaultOptions = &types.Options{
RandomAgent: false,
Metrics: false,
Debug: false,
DebugRequests: false,
+5 -1
View File
@@ -9,6 +9,10 @@ type Catalog struct {
// New creates a new Catalog structure using provided input items
func New(directory string) *Catalog {
catalog := &Catalog{templatesDirectory: directory}
catalog.readNucleiIgnoreFile()
return catalog
}
// AppendIgnore appends to the catalog store ignore list.
func (c *Catalog) AppendIgnore(list []string) {
c.ignoreFiles = append(c.ignoreFiles, list...)
}
+4 -5
View File
@@ -12,7 +12,7 @@ import (
)
// GetTemplatesPath returns a list of absolute paths for the provided template list.
func (c *Catalog) GetTemplatesPath(definitions []string) []string {
func (c *Catalog) GetTemplatesPath(definitions []string, noCheckIgnore bool) []string {
// keeps track of processed dirs and files
processed := make(map[string]bool)
allTemplates := []string{}
@@ -23,6 +23,9 @@ func (c *Catalog) GetTemplatesPath(definitions []string) []string {
gologger.Error().Msgf("Could not find template '%s': %s\n", t, err)
}
for _, path := range paths {
if !noCheckIgnore && c.checkIfInNucleiIgnore(path) {
continue
}
if _, ok := processed[path]; !ok {
processed[path] = true
allTemplates = append(allTemplates, path)
@@ -139,10 +142,6 @@ func (c *Catalog) findDirectoryMatches(absPath string, processed map[string]stru
},
Callback: func(path string, d *godirwalk.Dirent) error {
if !d.IsDir() && strings.HasSuffix(path, ".yaml") {
if c.checkIfInNucleiIgnore(path) {
return nil
}
if _, ok := processed[path]; !ok {
results = append(results, path)
processed[path] = struct{}{}
+1 -27
View File
@@ -1,37 +1,11 @@
package catalog
import (
"bufio"
"os"
"path"
"strings"
"github.com/projectdiscovery/gologger"
)
const nucleiIgnoreFile = ".nuclei-ignore"
// readNucleiIgnoreFile reads the nuclei ignore file marking it in map
func (c *Catalog) readNucleiIgnoreFile() {
file, err := os.Open(path.Join(c.templatesDirectory, nucleiIgnoreFile))
if err != nil {
return
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
text := scanner.Text()
if text == "" {
continue
}
if strings.HasPrefix(text, "#") {
continue
}
c.ignoreFiles = append(c.ignoreFiles, text)
}
}
// checkIfInNucleiIgnore checks if a path falls under nuclei-ignore rules.
func (c *Catalog) checkIfInNucleiIgnore(item string) bool {
if c.templatesDirectory == "" {
@@ -51,7 +25,7 @@ func (c *Catalog) checkIfInNucleiIgnore(item string) bool {
}
}
if matched {
gologger.Error().Msgf("Excluding %s due to nuclei-ignore filter", item)
gologger.Warning().Msgf("Excluding %s due to nuclei-ignore filter", item)
return true
}
return false
+25
View File
@@ -65,6 +65,30 @@ type Result struct {
PayloadValues map[string]interface{}
}
// Merge merges a result structure into the other.
func (r *Result) Merge(result *Result) {
if !r.Matched && result.Matched {
r.Matched = result.Matched
}
if !r.Extracted && result.Extracted {
r.Extracted = result.Extracted
}
for k, v := range result.Matches {
r.Matches[k] = v
}
for k, v := range result.Extracts {
r.Extracts[k] = v
}
r.OutputExtracts = append(r.OutputExtracts, result.OutputExtracts...)
for k, v := range result.DynamicValues {
r.DynamicValues[k] = v
}
for k, v := range result.PayloadValues {
r.PayloadValues[k] = v
}
}
// MatchFunc performs matching operation for a matcher on model and returns true or false.
type MatchFunc func(data map[string]interface{}, matcher *matchers.Matcher) bool
@@ -81,6 +105,7 @@ func (r *Operators) Execute(data map[string]interface{}, match MatchFunc, extrac
Extracts: make(map[string][]string),
DynamicValues: make(map[string]interface{}),
}
// Start with the extractors first and evaluate them.
for _, extractor := range r.Extractors {
var extractorResults []string
+7
View File
@@ -9,6 +9,7 @@ import (
jsoniter "github.com/json-iterator/go"
"github.com/logrusorgru/aurora"
"github.com/pkg/errors"
"github.com/projectdiscovery/interactsh/pkg/server"
"github.com/projectdiscovery/nuclei/v2/internal/colorizer"
"github.com/projectdiscovery/nuclei/v2/pkg/operators"
)
@@ -53,6 +54,8 @@ type InternalWrappedEvent struct {
type ResultEvent struct {
// TemplateID is the ID of the template for the result.
TemplateID string `json:"templateID"`
// TemplatePath is the path of template
TemplatePath string `json:"-"`
// Info contains information block of the template for the result.
Info map[string]interface{} `json:"info,inline"`
// MatcherName is the name of the matcher matched if any.
@@ -79,6 +82,10 @@ type ResultEvent struct {
IP string `json:"ip,omitempty"`
// Timestamp is the time the result was found at.
Timestamp time.Time `json:"timestamp"`
// Interaction is the full details of interactsh interaction.
Interaction *server.Interaction `json:"interaction,omitempty"`
FileToIndexPosition map[string]int `json:"-"`
}
// NewStandardWriter creates a new output writer based on user configurations
@@ -22,6 +22,7 @@ type Executer struct {
type clusteredOperator struct {
templateID string
templatePath string
templateInfo map[string]interface{}
operator *operators.Operators
}
@@ -38,6 +39,7 @@ func NewExecuter(requests []*templates.Template, options *protocols.ExecuterOpti
executer.operators = append(executer.operators, &clusteredOperator{
templateID: req.ID,
templateInfo: req.Info,
templatePath: req.Path,
operator: req.RequestsHTTP[0].CompiledOperators,
})
}
@@ -60,13 +62,15 @@ func (e *Executer) Requests() int {
func (e *Executer) Execute(input string) (bool, error) {
var results bool
previous := make(map[string]interface{})
dynamicValues := make(map[string]interface{})
err := e.requests.ExecuteWithResults(input, dynamicValues, nil, func(event *output.InternalWrappedEvent) {
err := e.requests.ExecuteWithResults(input, dynamicValues, previous, func(event *output.InternalWrappedEvent) {
for _, operator := range e.operators {
result, matched := operator.operator.Execute(event.InternalEvent, e.requests.Match, e.requests.Extract)
if matched && result != nil {
event.OperatorsResult = result
event.InternalEvent["template-id"] = operator.templateID
event.InternalEvent["template-path"] = operator.templatePath
event.InternalEvent["template-info"] = operator.templateInfo
event.Results = e.requests.MakeResultEvent(event)
results = true
@@ -94,6 +98,7 @@ func (e *Executer) ExecuteWithResults(input string, callback protocols.OutputEve
if matched && result != nil {
event.OperatorsResult = result
event.InternalEvent["template-id"] = operator.templateID
event.InternalEvent["template-path"] = operator.templatePath
event.InternalEvent["template-info"] = operator.templateInfo
event.Results = e.requests.MakeResultEvent(event)
callback(event)
@@ -0,0 +1,253 @@
package interactsh
import (
"net/url"
"strings"
"time"
"github.com/karlseguin/ccache"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/interactsh/pkg/client"
"github.com/projectdiscovery/interactsh/pkg/server"
"github.com/projectdiscovery/nuclei/v2/pkg/operators"
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/progress"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting"
"github.com/valyala/fasttemplate"
)
// Client is a wrapped client for interactsh server.
type Client struct {
// interactsh is a client for interactsh server.
interactsh *client.Client
// requests is a stored cache for interactsh-url->request-event data.
requests *ccache.Cache
// interactions is a stored cache for interactsh-interaction->interactsh-url data
interactions *ccache.Cache
options *Options
matched bool
dotHostname string
eviction time.Duration
pollDuration time.Duration
cooldownDuration time.Duration
}
var (
defaultInteractionDuration = 60 * time.Second
interactshURLMarker = "{{interactsh-url}}"
)
// Options contains configuration options for interactsh nuclei integration.
type Options struct {
// ServerURL is the URL of the interactsh server.
ServerURL string
// CacheSize is the numbers of requests to keep track of at a time.
// Older items are discarded in LRU manner in favor of new requests.
CacheSize int64
// Eviction is the period of time after which to automatically discard
// interaction requests.
Eviction time.Duration
// CooldownPeriod is additional time to wait for interactions after closing
// of the poller.
ColldownPeriod time.Duration
// PollDuration is the time to wait before each poll to the server for interactions.
PollDuration time.Duration
// Output is the output writer for nuclei
Output output.Writer
// IssuesClient is a client for issue exporting
IssuesClient *reporting.Client
// Progress is the nuclei progress bar implementation.
Progress progress.Progress
}
const defaultMaxInteractionsCount = 5000
// New returns a new interactsh server client
func New(options *Options) (*Client, error) {
parsed, err := url.Parse(options.ServerURL)
if err != nil {
return nil, errors.Wrap(err, "could not parse server url")
}
interactsh, err := client.New(&client.Options{
ServerURL: options.ServerURL,
PersistentSession: false,
})
if err != nil {
return nil, errors.Wrap(err, "could not create client")
}
configure := ccache.Configure()
configure = configure.MaxSize(options.CacheSize)
cache := ccache.New(configure)
interactionsCfg := ccache.Configure()
interactionsCfg = interactionsCfg.MaxSize(defaultMaxInteractionsCount)
interactionsCache := ccache.New(interactionsCfg)
interactClient := &Client{
interactsh: interactsh,
eviction: options.Eviction,
interactions: interactionsCache,
dotHostname: "." + parsed.Host,
options: options,
requests: cache,
pollDuration: options.PollDuration,
cooldownDuration: options.ColldownPeriod,
}
interactClient.interactsh.StartPolling(interactClient.pollDuration, func(interaction *server.Interaction) {
item := interactClient.requests.Get(interaction.UniqueID)
if item == nil {
// If we don't have any request for this ID, add it to temporary
// lru cache so we can correlate when we get an add request.
gotItem := interactClient.interactions.Get(interaction.UniqueID)
if gotItem == nil {
interactClient.interactions.Set(interaction.UniqueID, []*server.Interaction{interaction}, defaultInteractionDuration)
} else if items, ok := gotItem.Value().([]*server.Interaction); ok {
items = append(items, interaction)
interactClient.interactions.Set(interaction.UniqueID, items, defaultInteractionDuration)
}
return
}
request, ok := item.Value().(*RequestData)
if !ok {
return
}
_ = interactClient.processInteractionForRequest(interaction, request)
})
return interactClient, nil
}
// processInteractionForRequest processes an interaction for a request
func (c *Client) processInteractionForRequest(interaction *server.Interaction, data *RequestData) bool {
data.Event.InternalEvent["interactsh_protocol"] = interaction.Protocol
data.Event.InternalEvent["interactsh_request"] = interaction.RawRequest
data.Event.InternalEvent["interactsh_response"] = interaction.RawResponse
result, matched := data.Operators.Execute(data.Event.InternalEvent, data.MatchFunc, data.ExtractFunc)
if !matched || result == nil {
return false // if we don't match, return
}
c.requests.Delete(interaction.UniqueID)
if data.Event.OperatorsResult != nil {
data.Event.OperatorsResult.Merge(result)
} else {
data.Event.OperatorsResult = result
}
data.Event.Results = data.MakeResultFunc(data.Event)
for _, result := range data.Event.Results {
result.Interaction = interaction
_ = c.options.Output.Write(result)
if !c.matched {
c.matched = true
}
c.options.Progress.IncrementMatched()
if c.options.IssuesClient != nil {
if err := c.options.IssuesClient.CreateIssue(result); err != nil {
gologger.Warning().Msgf("Could not create issue on tracker: %s", err)
}
}
}
return true
}
// URL returns a new URL that can be interacted with
func (c *Client) URL() string {
return c.interactsh.URL()
}
// Close closes the interactsh clients after waiting for cooldown period.
func (c *Client) Close() bool {
if c.cooldownDuration > 0 {
time.Sleep(c.cooldownDuration)
}
c.interactsh.StopPolling()
c.interactsh.Close()
return c.matched
}
// ReplaceMarkers replaces the {{interactsh-url}} placeholders to actual
// URLs pointing to interactsh-server.
//
// It accepts data to replace as well as the URL to replace placeholders
// with generated uniquely for each request.
func (c *Client) ReplaceMarkers(data, interactshURL string) string {
if !strings.Contains(data, interactshURLMarker) {
return data
}
replaced := fasttemplate.ExecuteStringStd(data, "{{", "}}", map[string]interface{}{
"interactsh-url": interactshURL,
})
return replaced
}
// MakeResultEventFunc is a result making function for nuclei
type MakeResultEventFunc func(wrapped *output.InternalWrappedEvent) []*output.ResultEvent
// RequestData contains data for a request event
type RequestData struct {
MakeResultFunc MakeResultEventFunc
Event *output.InternalWrappedEvent
Operators *operators.Operators
MatchFunc operators.MatchFunc
ExtractFunc operators.ExtractFunc
}
// RequestEvent is the event for a network request sent by nuclei.
func (c *Client) RequestEvent(interactshURL string, data *RequestData) {
id := strings.TrimSuffix(interactshURL, c.dotHostname)
interaction := c.interactions.Get(id)
if interaction != nil {
// If we have previous interactions, get them and process them.
interactions, ok := interaction.Value().([]*server.Interaction)
if !ok {
c.requests.Set(id, data, c.eviction)
return
}
matched := false
for _, interaction := range interactions {
if c.processInteractionForRequest(interaction, data) {
matched = true
break
}
}
if matched {
c.interactions.Delete(id)
}
} else {
c.requests.Set(id, data, c.eviction)
}
}
// HasMatchers returns true if an operator has interactsh part
// matchers or extractors.
//
// Used by requests to show result or not depending on presence of interact.sh
// data part matchers.
func HasMatchers(op *operators.Operators) bool {
if op == nil {
return false
}
for _, matcher := range op.Matchers {
for _, dsl := range matcher.DSL {
if strings.Contains(dsl, "interactsh") {
return true
}
}
if strings.HasPrefix(matcher.Part, "interactsh") {
return true
}
}
for _, matcher := range op.Extractors {
if strings.HasPrefix(matcher.Part, "interactsh") {
return true
}
}
return false
}
@@ -1,6 +1,8 @@
package protocolinit
import (
"github.com/corpix/uarand"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/dns/dnsclientpool"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/network/networkclientpool"
@@ -9,6 +11,11 @@ import (
// Init initializes the client pools for the protocols
func Init(options *types.Options) error {
uarand.Default = uarand.NewWithCustomList(userAgents)
if err := protocolstate.Init(options); err != nil {
return err
}
if err := dnsclientpool.Init(options); err != nil {
return err
}
@@ -20,3 +27,42 @@ func Init(options *types.Options) error {
}
return nil
}
var userAgents = []string{
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36",
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2919.83 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2866.71 Safari/537.36",
"Mozilla/5.0 (X11; Ubuntu; Linux i686 on x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2820.59 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2762.73 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2656.18 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 4.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36",
"Mozilla/5.0 (X11; OpenBSD i386) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1944.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2117.157 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1866.237 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/4E423F",
}
func Close() {
protocolstate.Dialer.Close()
}
@@ -0,0 +1,31 @@
package protocolstate
import (
"github.com/pkg/errors"
"github.com/projectdiscovery/fastdialer/fastdialer"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
)
var Dialer *fastdialer.Dialer
func Init(options *types.Options) error {
opts := fastdialer.DefaultOptions
if options.SystemResolvers {
opts.EnableFallback = true
}
if options.ResolversFile != "" {
opts.BaseResolvers = options.InternalResolversList
}
dialer, err := fastdialer.NewDialer(opts)
if err != nil {
return errors.Wrap(err, "could not create dialer")
}
Dialer = dialer
return nil
}
func Close() {
if Dialer != nil {
Dialer.Close()
}
}
+2
View File
@@ -103,6 +103,7 @@ func (r *Request) responseToDSLMap(req, resp *dns.Msg, host, matched string) out
data["raw"] = rawData
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -137,6 +138,7 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "dns",
Host: types.ToString(wrapped.InternalEvent["host"]),
+1 -1
View File
@@ -37,7 +37,7 @@ type Request struct {
}
// defaultDenylist is the default list of extensions to be denied
var defaultDenylist = []string{".3g2", ".3gp", ".7z", ".apk", ".arj", ".avi", ".axd", ".bmp", ".css", ".csv", ".deb", ".dll", ".doc", ".drv", ".eot", ".exe", ".flv", ".gif", ".gifv", ".gz", ".h264", ".ico", ".iso", ".jar", ".jpeg", ".jpg", ".lock", ".m4a", ".m4v", ".map", ".mkv", ".mov", ".mp3", ".mp4", ".mpeg", ".mpg", ".msi", ".ogg", ".ogm", ".ogv", ".otf", ".pdf", ".pkg", ".png", ".ppt", ".psd", ".rar", ".rm", ".rpm", ".svg", ".swf", ".sys", ".tar.gz", ".tar", ".tif", ".tiff", ".ttf", ".txt", ".vob", ".wav", ".webm", ".wmv", ".woff", ".woff2", ".xcf", ".xls", ".xlsx", ".zip"}
var defaultDenylist = []string{".3g2", ".3gp", ".7z", ".apk", ".arj", ".avi", ".axd", ".bmp", ".css", ".csv", ".deb", ".dll", ".doc", ".drv", ".eot", ".exe", ".flv", ".gif", ".gifv", ".gz", ".h264", ".ico", ".iso", ".jar", ".jpeg", ".jpg", ".lock", ".m4a", ".m4v", ".map", ".mkv", ".mov", ".mp3", ".mp4", ".mpeg", ".mpg", ".msi", ".ogg", ".ogm", ".ogv", ".otf", ".pdf", ".pkg", ".png", ".ppt", ".psd", ".rar", ".rm", ".rpm", ".svg", ".swf", ".sys", ".tar.gz", ".tar", ".tif", ".tiff", ".ttf", ".vob", ".wav", ".webm", ".wmv", ".woff", ".woff2", ".xcf", ".xls", ".xlsx", ".zip"}
// GetID returns the unique ID of the request if any.
func (r *Request) GetID() string {
+32
View File
@@ -1,6 +1,8 @@
package file
import (
"bufio"
"strings"
"time"
"github.com/projectdiscovery/nuclei/v2/pkg/operators/extractors"
@@ -71,6 +73,7 @@ func (r *Request) responseToDSLMap(raw, host, matched string) output.InternalEve
data["raw"] = raw
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -99,16 +102,45 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
data := r.makeResultEventItem(wrapped)
results = append(results, data)
}
raw, ok := wrapped.InternalEvent["raw"]
if !ok {
return results
}
rawStr, ok := raw.(string)
if !ok {
return results
}
// Identify the position of match in file using a dirty hack.
for _, result := range results {
for _, extraction := range result.ExtractedResults {
scanner := bufio.NewScanner(strings.NewReader(rawStr))
line := 1
for scanner.Scan() {
if strings.Contains(scanner.Text(), extraction) {
if result.FileToIndexPosition == nil {
result.FileToIndexPosition = make(map[string]int)
}
result.FileToIndexPosition[result.Matched] = line
continue
}
line++
}
}
}
return results
}
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "file",
Path: types.ToString(wrapped.InternalEvent["path"]),
Matched: types.ToString(wrapped.InternalEvent["matched"]),
Host: types.ToString(wrapped.InternalEvent["matched"]),
ExtractedResults: wrapped.OperatorsResult.OutputExtracts,
Timestamp: time.Now(),
}
+3 -5
View File
@@ -73,13 +73,10 @@ func New(options *types.Options) (*Browser, error) {
customAgent = parts[1]
}
}
if options.RandomAgent {
if customAgent == "" {
customAgent = uarand.GetRandom()
}
httpclient, err := newhttpClient(options)
if err != nil {
return nil, err
}
httpclient := newhttpClient(options)
engine := &Browser{
tempDir: dataStore,
customAgent: customAgent,
@@ -102,6 +99,7 @@ func (b *Browser) Close() {
// headless process launch.
func (b *Browser) killChromeProcesses() {
newProcesses := b.findChromeProcesses()
for id := range newProcesses {
if _, ok := b.previouspids[id]; ok {
continue
@@ -5,25 +5,13 @@ import (
"net/http"
"time"
"github.com/pkg/errors"
"github.com/projectdiscovery/fastdialer/fastdialer"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
)
// newhttpClient creates a new http client for headless communication with a timeout
func newhttpClient(options *types.Options) (*http.Client, error) {
opts := fastdialer.DefaultOptions
if options.SystemResolvers {
opts.EnableFallback = true
}
if options.ResolversFile != "" {
opts.BaseResolvers = options.InternalResolversList
}
dialer, err := fastdialer.NewDialer(opts)
if err != nil {
return nil, errors.Wrap(err, "could not create dialer")
}
func newhttpClient(options *types.Options) *http.Client {
dialer := protocolstate.Dialer
transport := &http.Transport{
DialContext: dialer.Dial,
MaxIdleConns: 500,
@@ -34,5 +22,5 @@ func newhttpClient(options *types.Options) (*http.Client, error) {
InsecureSkipVerify: true,
},
}
return &http.Client{Transport: transport, Timeout: time.Duration(options.Timeout*3) * time.Second}, nil
return &http.Client{Transport: transport, Timeout: time.Duration(options.Timeout*3) * time.Second}
}
-1
View File
@@ -49,7 +49,6 @@ func (i *Instance) Run(baseURL *url.URL, actions []*Action, timeout time.Duratio
if err != nil {
return nil, nil, err
}
go router.Run()
data, err := createdPage.ExecuteActions(baseURL, actions)
if err != nil {
@@ -9,17 +9,21 @@ import (
"testing"
"time"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
"github.com/stretchr/testify/require"
)
func TestActionNavigate(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
instance, err := browser.NewInstance()
require.Nil(t, err, "could not create browser instance")
defer instance.Close()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, `
@@ -46,6 +50,8 @@ func TestActionNavigate(t *testing.T) {
}
func TestActionScript(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
@@ -113,6 +119,8 @@ func TestActionScript(t *testing.T) {
}
func TestActionClick(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
@@ -151,6 +159,8 @@ func TestActionClick(t *testing.T) {
}
func TestActionRightClick(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
@@ -197,6 +207,8 @@ func TestActionRightClick(t *testing.T) {
}
func TestActionTextInput(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
@@ -236,6 +248,8 @@ func TestActionTextInput(t *testing.T) {
}
func TestActionHeadersChange(t *testing.T) {
_ = protocolstate.Init(&types.Options{})
browser, err := New(&types.Options{ShowBrowser: false})
require.Nil(t, err, "could not create browser")
defer browser.Close()
+2
View File
@@ -72,6 +72,7 @@ func (r *Request) responseToDSLMap(resp, req, host, matched string) output.Inter
data["data"] = resp
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -106,6 +107,7 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "headless",
Host: types.ToString(wrapped.InternalEvent["host"]),
+36 -13
View File
@@ -11,6 +11,7 @@ import (
"strings"
"time"
"github.com/corpix/uarand"
"github.com/pkg/errors"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/expressions"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/generators"
@@ -36,7 +37,7 @@ type generatedRequest struct {
// Make creates a http request for the provided input.
// It returns io.EOF as error when all the requests have been exhausted.
func (r *requestGenerator) Make(baseURL string, dynamicValues map[string]interface{}) (*generatedRequest, error) {
func (r *requestGenerator) Make(baseURL string, dynamicValues map[string]interface{}, interactURL string) (*generatedRequest, error) {
// We get the next payload for the request.
data, payloads, ok := r.nextValue()
if !ok {
@@ -64,9 +65,9 @@ func (r *requestGenerator) Make(baseURL string, dynamicValues map[string]interfa
// If data contains \n it's a raw request, process it like raw. Else
// continue with the template based request flow.
if isRawRequest {
return r.makeHTTPRequestFromRaw(ctx, parsedString, data, values, payloads)
return r.makeHTTPRequestFromRaw(ctx, parsedString, data, values, payloads, interactURL)
}
return r.makeHTTPRequestFromModel(ctx, data, values)
return r.makeHTTPRequestFromModel(ctx, data, values, interactURL)
}
// Total returns the total number of requests for the generator
@@ -95,8 +96,11 @@ func baseURLWithTemplatePrefs(data string, parsed *url.URL) (string, *url.URL) {
}
// MakeHTTPRequestFromModel creates a *http.Request from a request template
func (r *requestGenerator) makeHTTPRequestFromModel(ctx context.Context, data string, values map[string]interface{}) (*generatedRequest, error) {
func (r *requestGenerator) makeHTTPRequestFromModel(ctx context.Context, data string, values map[string]interface{}, interactURL string) (*generatedRequest, error) {
final := replacer.Replace(data, values)
if interactURL != "" {
final = r.options.Interactsh.ReplaceMarkers(final, interactURL)
}
// Build a request on the specified URL
req, err := http.NewRequestWithContext(ctx, r.request.Method, final, nil)
@@ -104,7 +108,7 @@ func (r *requestGenerator) makeHTTPRequestFromModel(ctx context.Context, data st
return nil, err
}
request, err := r.fillRequest(req, values)
request, err := r.fillRequest(req, values, interactURL)
if err != nil {
return nil, err
}
@@ -112,12 +116,15 @@ func (r *requestGenerator) makeHTTPRequestFromModel(ctx context.Context, data st
}
// makeHTTPRequestFromRaw creates a *http.Request from a raw request
func (r *requestGenerator) makeHTTPRequestFromRaw(ctx context.Context, baseURL, data string, values, payloads map[string]interface{}) (*generatedRequest, error) {
return r.handleRawWithPaylods(ctx, data, baseURL, values, payloads)
func (r *requestGenerator) makeHTTPRequestFromRaw(ctx context.Context, baseURL, data string, values, payloads map[string]interface{}, interactURL string) (*generatedRequest, error) {
if interactURL != "" {
data = r.options.Interactsh.ReplaceMarkers(data, interactURL)
}
return r.handleRawWithPayloads(ctx, data, baseURL, values, payloads)
}
// handleRawWithPaylods handles raw requests along with paylaods
func (r *requestGenerator) handleRawWithPaylods(ctx context.Context, rawRequest, baseURL string, values, generatorValues map[string]interface{}) (*generatedRequest, error) {
// handleRawWithPayloads handles raw requests along with payloads
func (r *requestGenerator) handleRawWithPayloads(ctx context.Context, rawRequest, baseURL string, values, generatorValues map[string]interface{}) (*generatedRequest, error) {
// Combine the template payloads along with base
// request values.
finalValues := generators.MergeMaps(generatorValues, values)
@@ -157,8 +164,11 @@ func (r *requestGenerator) handleRawWithPaylods(ctx context.Context, rawRequest,
continue
}
req.Header[key] = []string{value}
if key == "Host" {
req.Host = value
}
}
request, err := r.fillRequest(req, values)
request, err := r.fillRequest(req, values, "")
if err != nil {
return nil, err
}
@@ -167,10 +177,16 @@ func (r *requestGenerator) handleRawWithPaylods(ctx context.Context, rawRequest,
}
// fillRequest fills various headers in the request with values
func (r *requestGenerator) fillRequest(req *http.Request, values map[string]interface{}) (*retryablehttp.Request, error) {
func (r *requestGenerator) fillRequest(req *http.Request, values map[string]interface{}, interactURL string) (*retryablehttp.Request, error) {
// Set the header values requested
for header, value := range r.request.Headers {
if interactURL != "" {
value = r.options.Interactsh.ReplaceMarkers(value, interactURL)
}
req.Header[header] = []string{replacer.Replace(value, values)}
if header == "Host" {
req.Host = replacer.Replace(value, values)
}
}
// In case of multiple threads the underlying connection should remain open to allow reuse
@@ -180,9 +196,13 @@ func (r *requestGenerator) fillRequest(req *http.Request, values map[string]inte
// Check if the user requested a request body
if r.request.Body != "" {
req.Body = ioutil.NopCloser(strings.NewReader(r.request.Body))
body := r.request.Body
if interactURL != "" {
body = r.options.Interactsh.ReplaceMarkers(body, interactURL)
}
req.Body = ioutil.NopCloser(strings.NewReader(body))
}
setHeader(req, "User-Agent", "Nuclei - Open-source project (github.com/projectdiscovery/nuclei)")
setHeader(req, "User-Agent", uarand.GetRandom())
// Only set these headers on non raw requests
if len(r.request.Raw) == 0 {
@@ -197,4 +217,7 @@ func setHeader(req *http.Request, name, value string) {
if _, ok := req.Header[name]; !ok {
req.Header.Set(name, value)
}
if name == "Host" {
req.Host = value
}
}
+7 -7
View File
@@ -42,7 +42,7 @@ func TestMakeRequestFromModal(t *testing.T) {
require.Nil(t, err, "could not compile http request")
generator := request.newGenerator()
req, err := generator.Make("https://example.com", map[string]interface{}{})
req, err := generator.Make("https://example.com", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
bodyBytes, _ := req.request.BodyBytes()
@@ -69,12 +69,12 @@ func TestMakeRequestFromModalTrimSuffixSlash(t *testing.T) {
require.Nil(t, err, "could not compile http request")
generator := request.newGenerator()
req, err := generator.Make("https://example.com/test.php", map[string]interface{}{})
req, err := generator.Make("https://example.com/test.php", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
require.Equal(t, "https://example.com/test.php?query=example", req.request.URL.String(), "could not get correct request path")
generator = request.newGenerator()
req, err = generator.Make("https://example.com/test/", map[string]interface{}{})
req, err = generator.Make("https://example.com/test/", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
require.Equal(t, "https://example.com/test/?query=example", req.request.URL.String(), "could not get correct request path")
}
@@ -107,12 +107,12 @@ Accept-Encoding: gzip`},
require.Nil(t, err, "could not compile http request")
generator := request.newGenerator()
req, err := generator.Make("https://example.com", map[string]interface{}{})
req, err := generator.Make("https://example.com", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
authorization := req.request.Header.Get("Authorization")
require.Equal(t, "Basic admin:admin", authorization, "could not get correct authorization headers from raw")
req, err = generator.Make("https://example.com", map[string]interface{}{})
req, err = generator.Make("https://example.com", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
authorization = req.request.Header.Get("Authorization")
require.Equal(t, "Basic admin:guest", authorization, "could not get correct authorization headers from raw")
@@ -146,12 +146,12 @@ Accept-Encoding: gzip`},
require.Nil(t, err, "could not compile http request")
generator := request.newGenerator()
req, err := generator.Make("https://example.com", map[string]interface{}{})
req, err := generator.Make("https://example.com", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
authorization := req.request.Header.Get("Authorization")
require.Equal(t, "Basic YWRtaW46YWRtaW4=", authorization, "could not get correct authorization headers from raw")
req, err = generator.Make("https://example.com", map[string]interface{}{})
req, err = generator.Make("https://example.com", map[string]interface{}{}, "")
require.Nil(t, err, "could not make http request")
authorization = req.request.Header.Get("Authorization")
require.Equal(t, "Basic YWRtaW46Z3Vlc3Q=", authorization, "could not get correct authorization headers from raw")
+1 -1
View File
@@ -10,7 +10,7 @@ import (
// are similar enough to be considered one and can be checked by
// just adding the matcher/extractors for the request and the correct IDs.
func (r *Request) CanCluster(other *Request) bool {
if len(r.Payloads) > 0 || len(r.Raw) > 0 || len(r.Body) > 0 || r.Unsafe {
if len(r.Payloads) > 0 || len(r.Raw) > 0 || len(r.Body) > 0 || r.Unsafe || r.ReqCondition || r.Name != "" {
return false
}
if r.Method != other.Method ||
-5
View File
@@ -3,7 +3,6 @@ package http
import (
"strings"
"github.com/corpix/uarand"
"github.com/pkg/errors"
"github.com/projectdiscovery/nuclei/v2/pkg/operators"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols"
@@ -102,10 +101,6 @@ func (r *Request) Compile(options *protocols.ExecuterOptions) error {
}
r.customHeaders[parts[0]] = strings.TrimSpace(parts[1])
}
// Add User-Agent value randomly to the customHeaders slice if `random-agent` flag is given
if r.options.Options.RandomAgent {
r.customHeaders["User-Agent"] = uarand.GetRandom()
}
if r.Body != "" && !strings.Contains(r.Body, "\r\n") {
r.Body = strings.ReplaceAll(r.Body, "\n", "\r\n")
@@ -15,6 +15,7 @@ import (
"github.com/pkg/errors"
"github.com/projectdiscovery/fastdialer/fastdialer"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
"github.com/projectdiscovery/rawhttp"
"github.com/projectdiscovery/retryablehttp-go"
@@ -97,14 +98,7 @@ func wrappedGet(options *types.Options, configuration *Configuration) (*retryabl
var err error
if Dialer == nil {
opts := fastdialer.DefaultOptions
if options.SystemResolvers {
opts.EnableFallback = true
}
if options.ResolversFile != "" {
opts.BaseResolvers = options.InternalResolversList
}
Dialer, err = fastdialer.NewDialer(opts)
Dialer = protocolstate.Dialer
}
if err != nil {
return nil, errors.Wrap(err, "could not create dialer")
+2
View File
@@ -105,6 +105,7 @@ func (r *Request) responseToDSLMap(resp *http.Response, host, matched, rawReq, r
data["duration"] = duration.Seconds()
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -139,6 +140,7 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "http",
Host: types.ToString(wrapped.InternalEvent["host"]),
+6 -10
View File
@@ -5,7 +5,6 @@ import (
"fmt"
"io"
"io/ioutil"
"net"
"net/url"
"strings"
@@ -100,20 +99,11 @@ func Parse(request, baseURL string, unsafe bool) (*Request, error) {
rawRequest.Path = parts[1]
}
// If raw request doesn't have a Host header and/ path,
// this will be generated from the parsed baseURL
parsedURL, err := url.Parse(baseURL)
if err != nil {
return nil, fmt.Errorf("could not parse request URL: %s", err)
}
templateHost := rawRequest.Headers["Host"]
hostURL := parsedURL.Host
if strings.Contains(templateHost, ":") {
_, templatePort, _ := net.SplitHostPort(templateHost)
hostURL = net.JoinHostPort(parsedURL.Hostname(), templatePort)
}
if strings.HasSuffix(parsedURL.Path, "/") && strings.HasPrefix(rawRequest.Path, "/") {
parsedURL.Path = strings.TrimSuffix(parsedURL.Path, "/")
}
@@ -123,6 +113,12 @@ func Parse(request, baseURL string, unsafe bool) (*Request, error) {
}
rawRequest.FullURL = fmt.Sprintf("%s://%s%s", parsedURL.Scheme, strings.TrimSpace(hostURL), rawRequest.Path)
// If raw request doesn't have a Host header
// this will be generated from the parsed baseURL
if rawRequest.Headers["Host"] == "" {
rawRequest.Headers["Host"] = hostURL
}
// Set the request body
b, err := ioutil.ReadAll(reader)
if err != nil {
+4 -4
View File
@@ -15,7 +15,7 @@ User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Language: en-US,en;q=0.9`, "https://example.com:8080", false)
require.Nil(t, err, "could not parse GET request")
require.Equal(t, "https://example.com:123/gg/phpinfo.php", request.FullURL, "Could not parse request url correctly")
require.Equal(t, "https://example.com:8080/gg/phpinfo.php", request.FullURL, "Could not parse request url correctly")
require.Equal(t, "/gg/phpinfo.php", request.Path, "Could not parse request path correctly")
t.Run("path-suffix", func(t *testing.T) {
@@ -29,17 +29,17 @@ Host: {{Hostname}}`, "https://example.com:8080/test", false)
request, err := Parse(`GET ?username=test&password=test HTTP/1.1
Host: {{Hostname}}:123`, "https://example.com:8080/test", false)
require.Nil(t, err, "could not parse GET request")
require.Equal(t, "https://example.com:123/test?username=test&password=test", request.FullURL, "Could not parse request url correctly")
require.Equal(t, "https://example.com:8080/test?username=test&password=test", request.FullURL, "Could not parse request url correctly")
request, err = Parse(`GET ?username=test&password=test HTTP/1.1
Host: {{Hostname}}:123`, "https://example.com:8080/test/", false)
require.Nil(t, err, "could not parse GET request")
require.Equal(t, "https://example.com:123/test/?username=test&password=test", request.FullURL, "Could not parse request url correctly")
require.Equal(t, "https://example.com:8080/test/?username=test&password=test", request.FullURL, "Could not parse request url correctly")
request, err = Parse(`GET /?username=test&password=test HTTP/1.1
Host: {{Hostname}}:123`, "https://example.com:8080/test/", false)
require.Nil(t, err, "could not parse GET request")
require.Equal(t, "https://example.com:123/test/?username=test&password=test", request.FullURL, "Could not parse request url correctly")
require.Equal(t, "https://example.com:8080/test/?username=test&password=test", request.FullURL, "Could not parse request url correctly")
})
}
+52 -20
View File
@@ -17,6 +17,7 @@ import (
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/tostring"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/rawhttp"
@@ -33,7 +34,7 @@ func (r *Request) executeRaceRequest(reqURL string, previous output.InternalEven
// Requests within race condition should be dumped once and the output prefilled to allow DSL language to work
// This will introduce a delay and will populate in hacky way the field "request" of outputEvent
generator := r.newGenerator()
requestForDump, err := generator.Make(reqURL, nil)
requestForDump, err := generator.Make(reqURL, nil, "")
if err != nil {
return err
}
@@ -51,7 +52,7 @@ func (r *Request) executeRaceRequest(reqURL string, previous output.InternalEven
// Pre-Generate requests
for i := 0; i < r.RaceNumberRequests; i++ {
generator := r.newGenerator()
request, err := generator.Make(reqURL, nil)
request, err := generator.Make(reqURL, nil, "")
if err != nil {
return err
}
@@ -90,7 +91,7 @@ func (r *Request) executeParallelHTTP(reqURL string, dynamicValues, previous out
var requestErr error
mutex := &sync.Mutex{}
for {
request, err := generator.Make(reqURL, dynamicValues)
request, err := generator.Make(reqURL, dynamicValues, "")
if err == io.EOF {
break
}
@@ -116,7 +117,7 @@ func (r *Request) executeParallelHTTP(reqURL string, dynamicValues, previous out
return requestErr
}
// executeRaceRequest executes turbo http request for a URL
// executeTurboHTTP executes turbo http request for a URL
func (r *Request) executeTurboHTTP(reqURL string, dynamicValues, previous output.InternalEvent, callback protocols.OutputEventCallback) error {
generator := r.newGenerator()
@@ -148,7 +149,7 @@ func (r *Request) executeTurboHTTP(reqURL string, dynamicValues, previous output
var requestErr error
mutex := &sync.Mutex{}
for {
request, err := generator.Make(reqURL, dynamicValues)
request, err := generator.Make(reqURL, dynamicValues, "")
if err == io.EOF {
break
}
@@ -197,7 +198,13 @@ func (r *Request) ExecuteWithResults(reqURL string, dynamicValues, previous outp
requestCount := 1
var requestErr error
for {
request, err := generator.Make(reqURL, dynamicValues)
hasInteractMarkers := interactsh.HasMatchers(r.CompiledOperators)
var interactURL string
if r.options.Interactsh != nil && hasInteractMarkers {
interactURL = r.options.Interactsh.URL()
}
request, err := generator.Make(reqURL, dynamicValues, interactURL)
if err == io.EOF {
break
}
@@ -214,7 +221,17 @@ func (r *Request) ExecuteWithResults(reqURL string, dynamicValues, previous outp
gotOutput = true
dynamicValues = generators.MergeMaps(dynamicValues, event.OperatorsResult.DynamicValues)
}
callback(event)
if hasInteractMarkers && r.options.Interactsh != nil {
r.options.Interactsh.RequestEvent(interactURL, &interactsh.RequestData{
MakeResultFunc: r.MakeResultEvent,
Event: event,
Operators: r.CompiledOperators,
MatchFunc: r.Match,
ExtractFunc: r.Extract,
})
} else {
callback(event)
}
}, requestCount)
if err != nil {
requestErr = multierr.Append(requestErr, err)
@@ -260,11 +277,15 @@ func (r *Request) executeRequest(reqURL string, request *generatedRequest, previ
var hostname string
timeStart := time.Now()
if request.original.Pipeline {
formedURL = request.rawRequest.FullURL
if parsed, parseErr := url.Parse(formedURL); parseErr == nil {
hostname = parsed.Host
if request.rawRequest != nil {
formedURL = request.rawRequest.FullURL
if parsed, parseErr := url.Parse(formedURL); parseErr == nil {
hostname = parsed.Host
}
resp, err = request.pipelinedClient.DoRaw(request.rawRequest.Method, reqURL, request.rawRequest.Path, generators.ExpandMapValues(request.rawRequest.Headers), ioutil.NopCloser(strings.NewReader(request.rawRequest.Data)))
} else if request.request != nil {
resp, err = request.pipelinedClient.Dor(request.request)
}
resp, err = request.pipelinedClient.DoRaw(request.rawRequest.Method, reqURL, request.rawRequest.Path, generators.ExpandMapValues(request.rawRequest.Headers), ioutil.NopCloser(strings.NewReader(request.rawRequest.Data)))
} else if request.original.Unsafe && request.rawRequest != nil {
formedURL = request.rawRequest.FullURL
if parsed, parseErr := url.Parse(formedURL); parseErr == nil {
@@ -326,7 +347,9 @@ func (r *Request) executeRequest(reqURL string, request *generatedRequest, previ
}
data, err := ioutil.ReadAll(bodyReader)
if err != nil {
return errors.Wrap(err, "could not read http body")
if !strings.Contains(err.Error(), "unexpected EOF") { // ignore EOF error
return errors.Wrap(err, "could not read http body")
}
}
resp.Body.Close()
@@ -372,6 +395,9 @@ func (r *Request) executeRequest(reqURL string, request *generatedRequest, previ
finalEvent := make(output.InternalEvent)
outputEvent := r.responseToDSLMap(resp, reqURL, matchedURL, tostring.UnsafeToString(dumpedRequest), tostring.UnsafeToString(dumpedResponse), tostring.UnsafeToString(data), headersToString(resp.Header), duration, request.meta)
if i := strings.LastIndex(hostname, ":"); i != -1 {
hostname = hostname[:i]
}
outputEvent["ip"] = httpclientpool.Dialer.GetDialedIP(hostname)
outputEvent["redirect-chain"] = tostring.UnsafeToString(redirectedResponse)
for k, v := range previous {
@@ -390,14 +416,16 @@ func (r *Request) executeRequest(reqURL string, request *generatedRequest, previ
}
event := &output.InternalWrappedEvent{InternalEvent: outputEvent}
if r.CompiledOperators != nil {
var ok bool
event.OperatorsResult, ok = r.CompiledOperators.Execute(finalEvent, r.Match, r.Extract)
if ok && event.OperatorsResult != nil {
event.OperatorsResult.PayloadValues = request.meta
event.Results = r.MakeResultEvent(event)
if !interactsh.HasMatchers(r.CompiledOperators) {
if r.CompiledOperators != nil {
var ok bool
event.OperatorsResult, ok = r.CompiledOperators.Execute(finalEvent, r.Match, r.Extract)
if ok && event.OperatorsResult != nil {
event.OperatorsResult.PayloadValues = request.meta
event.Results = r.MakeResultEvent(event)
}
event.InternalEvent = outputEvent
}
event.InternalEvent = outputEvent
}
callback(event)
return nil
@@ -409,7 +437,11 @@ func (r *Request) setCustomHeaders(req *generatedRequest) {
if req.rawRequest != nil {
req.rawRequest.Headers[k] = v
} else {
req.request.Header.Set(strings.TrimSpace(k), strings.TrimSpace(v))
kk, vv := strings.TrimSpace(k), strings.TrimSpace(v)
req.request.Header.Set(kk, vv)
if kk == "Host" {
req.request.Host = vv
}
}
}
}
+6 -2
View File
@@ -1,6 +1,9 @@
package http
import "github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/generators"
import (
"github.com/projectdiscovery/nuclei/v2/pkg/protocols"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/generators"
)
// requestGenerator generates requests sequentially based on various
// configurations for a http request template.
@@ -11,12 +14,13 @@ import "github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/generators"
type requestGenerator struct {
currentIndex int
request *Request
options *protocols.ExecuterOptions
payloadIterator *generators.Iterator
}
// newGenerator creates a new request generator instance
func (r *Request) newGenerator() *requestGenerator {
generator := &requestGenerator{request: r}
generator := &requestGenerator{request: r, options: r.options}
if len(r.Payloads) > 0 {
generator.payloadIterator = r.generator.NewIterator()
+20 -14
View File
@@ -3,6 +3,8 @@ package http
import (
"bytes"
"compress/gzip"
"compress/zlib"
"io"
"io/ioutil"
"net/http"
"net/http/httputil"
@@ -100,19 +102,23 @@ func handleDecompression(resp *http.Response, bodyOrig []byte) (bodyDec []byte,
return bodyOrig, nil
}
encodingHeader := strings.TrimSpace(strings.ToLower(resp.Header.Get("Content-Encoding")))
if strings.Contains(encodingHeader, "gzip") {
gzipreader, err := gzip.NewReader(bytes.NewReader(bodyOrig))
if err != nil {
return bodyOrig, err
}
defer gzipreader.Close()
bodyDec, err = ioutil.ReadAll(gzipreader)
if err != nil {
return bodyOrig, err
}
return bodyDec, nil
var reader io.ReadCloser
switch resp.Header.Get("Content-Encoding") {
case "gzip":
reader, err = gzip.NewReader(resp.Body)
case "deflate":
reader, err = zlib.NewReader(resp.Body)
default:
reader = resp.Body
}
return bodyOrig, nil
if err != nil {
return nil, err
}
defer reader.Close()
bodyDec, err = ioutil.ReadAll(reader)
if err != nil {
return bodyOrig, err
}
return bodyDec, nil
}
@@ -1,8 +1,8 @@
package networkclientpool
import (
"github.com/pkg/errors"
"github.com/projectdiscovery/fastdialer/fastdialer"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
)
@@ -16,18 +16,7 @@ func Init(options *types.Options) error {
if normalClient != nil {
return nil
}
opts := fastdialer.DefaultOptions
if options.SystemResolvers {
opts.EnableFallback = true
}
if options.ResolversFile != "" {
opts.BaseResolvers = options.InternalResolversList
}
dialer, err := fastdialer.NewDialer(opts)
if err != nil {
return errors.Wrap(err, "could not create dialer")
}
normalClient = dialer
normalClient = protocolstate.Dialer
return nil
}
+2
View File
@@ -73,6 +73,7 @@ func (r *Request) responseToDSLMap(req, resp, raw, host, matched string) output.
data["raw"] = raw // Raw is the full transaction data for network
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -107,6 +108,7 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "network",
Host: types.ToString(wrapped.InternalEvent["host"]),
+26 -6
View File
@@ -13,6 +13,7 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/replacer"
)
@@ -77,6 +78,12 @@ func (r *Request) executeAddress(actualAddress, address, input string, shouldUse
defer conn.Close()
_ = conn.SetReadDeadline(time.Now().Add(time.Duration(r.options.Options.Timeout) * time.Second))
hasInteractMarkers := interactsh.HasMatchers(r.CompiledOperators)
var interactURL string
if r.options.Interactsh != nil && hasInteractMarkers {
interactURL = r.options.Interactsh.URL()
}
responseBuilder := &strings.Builder{}
reqBuilder := &strings.Builder{}
@@ -88,6 +95,9 @@ func (r *Request) executeAddress(actualAddress, address, input string, shouldUse
case "hex":
data, err = hex.DecodeString(input.Data)
default:
if interactURL != "" {
input.Data = r.options.Interactsh.ReplaceMarkers(input.Data, interactURL)
}
data = []byte(input.Data)
}
if err != nil {
@@ -150,14 +160,24 @@ func (r *Request) executeAddress(actualAddress, address, input string, shouldUse
}
event := &output.InternalWrappedEvent{InternalEvent: outputEvent}
if r.CompiledOperators != nil {
result, ok := r.CompiledOperators.Execute(outputEvent, r.Match, r.Extract)
if ok && result != nil {
event.OperatorsResult = result
event.Results = r.MakeResultEvent(event)
if !hasInteractMarkers {
if r.CompiledOperators != nil {
result, ok := r.CompiledOperators.Execute(outputEvent, r.Match, r.Extract)
if ok && result != nil {
event.OperatorsResult = result
event.Results = r.MakeResultEvent(event)
}
}
callback(event)
} else if r.options.Interactsh != nil {
r.options.Interactsh.RequestEvent(interactURL, &interactsh.RequestData{
MakeResultFunc: r.MakeResultEvent,
Event: event,
Operators: r.CompiledOperators,
MatchFunc: r.Match,
ExtractFunc: r.Extract,
})
}
callback(event)
return nil
}
@@ -101,6 +101,7 @@ func (r *Request) responseToDSLMap(resp *http.Response, host, matched, rawReq, r
data["duration"] = duration.Seconds()
data["template-id"] = r.options.TemplateID
data["template-info"] = r.options.TemplateInfo
data["template-path"] = r.options.TemplatePath
return data
}
@@ -135,6 +136,7 @@ func (r *Request) MakeResultEvent(wrapped *output.InternalWrappedEvent) []*outpu
func (r *Request) makeResultEventItem(wrapped *output.InternalWrappedEvent) *output.ResultEvent {
data := &output.ResultEvent{
TemplateID: types.ToString(wrapped.InternalEvent["template-id"]),
TemplatePath: types.ToString(wrapped.InternalEvent["template-path"]),
Info: wrapped.InternalEvent["template-info"].(map[string]interface{}),
Type: "http",
Path: types.ToString(wrapped.InternalEvent["path"]),
+3
View File
@@ -8,6 +8,7 @@ import (
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/progress"
"github.com/projectdiscovery/nuclei/v2/pkg/projectfile"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v2/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting"
"github.com/projectdiscovery/nuclei/v2/pkg/types"
@@ -50,6 +51,8 @@ type ExecuterOptions struct {
ProjectFile *projectfile.ProjectFile
// Browser is a browser engine for running headless templates
Browser *engine.Browser
// Interactsh is a client for interactsh oob polling server
Interactsh *interactsh.Client
Operators []*operators.Operators // only used by offlinehttp module
}
+5
View File
@@ -58,3 +58,8 @@ func (i *Exporter) Export(event *output.ResultEvent) error {
err := ioutil.WriteFile(path.Join(i.directory, finalFilename), data, 0644)
return err
}
// Close closes the exporter after operation
func (i *Exporter) Close() error {
return nil
}
+144
View File
@@ -0,0 +1,144 @@
package sarif
import (
"crypto/sha1"
"encoding/hex"
"os"
"path"
"strings"
"sync"
"github.com/owenrumney/go-sarif/sarif"
"github.com/pkg/errors"
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/format"
)
// Exporter is an exporter for nuclei sarif output format.
type Exporter struct {
sarif *sarif.Report
run *sarif.Run
mutex *sync.Mutex
home string
tempFile string
options *Options
}
// Options contains the configuration options for sarif exporter client
type Options struct {
// File is the file to export found sarif result to
File string `yaml:"file"`
}
// New creates a new disk exporter integration client based on options.
func New(options *Options) (*Exporter, error) {
report, err := sarif.New(sarif.Version210)
if err != nil {
return nil, errors.Wrap(err, "could not create sarif exporter")
}
home, err := os.UserHomeDir()
if err != nil {
return nil, errors.Wrap(err, "could not get home dir")
}
templatePath := path.Join(home, "nuclei-templates")
run := sarif.NewRun("nuclei", "https://github.com/projectdiscovery/nuclei")
return &Exporter{options: options, home: templatePath, sarif: report, run: run, mutex: &sync.Mutex{}}, nil
}
// Export exports a passed result event to sarif structure
func (i *Exporter) Export(event *output.ResultEvent) error {
templatePath := strings.TrimPrefix(event.TemplatePath, i.home)
h := sha1.New()
h.Write([]byte(event.Host))
templateID := event.TemplateID + "-" + hex.EncodeToString(h.Sum(nil))
fullDescription := format.MarkdownDescription(event)
sarifSeverity := getSarifSeverity(event)
var ruleName string
if s, ok := event.Info["name"]; ok {
ruleName = s.(string)
}
var templateURL string
if strings.HasPrefix(event.TemplatePath, i.home) {
templateURL = "https://github.com/projectdiscovery/nuclei-templates/blob/master" + templatePath
} else {
templateURL = "https://github.com/projectdiscovery/nuclei-templates"
}
var ruleDescription string
if d, ok := event.Info["description"]; ok {
ruleDescription = d.(string)
}
i.mutex.Lock()
defer i.mutex.Unlock()
_ = i.run.AddRule(templateID).
WithDescription(ruleName).
WithHelp(fullDescription).
WithHelpURI(templateURL).
WithFullDescription(sarif.NewMultiformatMessageString(ruleDescription))
result := i.run.AddResult(templateID).
WithMessage(sarif.NewMessage().WithText(event.Host)).
WithLevel(sarifSeverity)
// Also write file match metadata to file
if event.Type == "file" && (event.FileToIndexPosition != nil && len(event.FileToIndexPosition) > 0) {
for file, line := range event.FileToIndexPosition {
result.WithLocation(sarif.NewLocation().WithMessage(sarif.NewMessage().WithText(ruleName)).WithPhysicalLocation(
sarif.NewPhysicalLocation().
WithArtifactLocation(sarif.NewArtifactLocation().WithUri(file)).
WithRegion(sarif.NewRegion().WithStartColumn(1).WithStartLine(line).WithEndLine(line).WithEndColumn(32)),
))
}
} else {
result.WithLocation(sarif.NewLocation().WithMessage(sarif.NewMessage().WithText(event.Host)).WithPhysicalLocation(
sarif.NewPhysicalLocation().
WithArtifactLocation(sarif.NewArtifactLocation().WithUri("README.md")).
WithRegion(sarif.NewRegion().WithStartColumn(1).WithStartLine(1).WithEndLine(1).WithEndColumn(1)),
))
}
return nil
}
// getSarifSeverity returns the sarif severity
func getSarifSeverity(event *output.ResultEvent) string {
var ruleSeverity string
if s, ok := event.Info["severity"]; ok {
ruleSeverity = s.(string)
}
switch ruleSeverity {
case "info":
return "note"
case "low", "medium":
return "warning"
case "high", "critical":
return "error"
default:
return "note"
}
}
// Close closes the exporter after operation
func (i *Exporter) Close() error {
i.mutex.Lock()
defer i.mutex.Unlock()
i.sarif.AddRun(i.run)
if len(i.run.Results) == 0 {
return nil // do not write when no results
}
file, err := os.Create(i.options.File)
if err != nil {
return errors.Wrap(err, "could not create sarif output file")
}
defer file.Close()
return i.sarif.Write(file)
}
+50 -2
View File
@@ -44,6 +44,9 @@ func MarkdownDescription(event *output.ResultEvent) string {
builder.WriteString(event.Timestamp.Format("Mon Jan 2 15:04:05 -0700 MST 2006"))
builder.WriteString("\n\n**Template Information**\n\n| Key | Value |\n|---|---|\n")
for k, v := range event.Info {
if k == "reference" {
continue
}
builder.WriteString(fmt.Sprintf("| %s | %s |\n", k, v))
}
if event.Request != "" {
@@ -60,11 +63,11 @@ func MarkdownDescription(event *output.ResultEvent) string {
} else {
builder.WriteString(event.Response)
}
builder.WriteString("\n```\n\n")
builder.WriteString("\n```\n")
}
if len(event.ExtractedResults) > 0 || len(event.Metadata) > 0 {
builder.WriteString("**Extra Information**\n\n")
builder.WriteString("\n**Extra Information**\n\n")
if len(event.ExtractedResults) > 0 {
builder.WriteString("**Extracted results**:\n\n")
for _, v := range event.ExtractedResults {
@@ -86,6 +89,51 @@ func MarkdownDescription(event *output.ResultEvent) string {
builder.WriteString("\n")
}
}
if event.Interaction != nil {
builder.WriteString("**Interaction Data**\n---\n")
builder.WriteString(event.Interaction.Protocol)
if event.Interaction.QType != "" {
builder.WriteString(" (")
builder.WriteString(event.Interaction.QType)
builder.WriteString(")")
}
builder.WriteString(" Interaction from ")
builder.WriteString(event.Interaction.RemoteAddress)
builder.WriteString(" at ")
builder.WriteString(event.Interaction.UniqueID)
if event.Interaction.RawRequest != "" {
builder.WriteString("\n\n**Interaction Request**\n\n```\n")
builder.WriteString(event.Interaction.RawRequest)
builder.WriteString("\n```\n")
}
if event.Interaction.RawResponse != "" {
builder.WriteString("\n**Interaction Response**\n\n```\n")
builder.WriteString(event.Interaction.RawResponse)
builder.WriteString("\n```\n")
}
}
if d, ok := event.Info["reference"]; ok {
builder.WriteString("\nReference: \n")
switch v := d.(type) {
case string:
if !strings.HasPrefix(v, "-") {
builder.WriteString("- ")
}
builder.WriteString(v)
case []interface{}:
slice := types.ToStringSlice(v)
for i, item := range slice {
builder.WriteString("- ")
builder.WriteString(item)
if len(slice)-1 != i {
builder.WriteString("\n")
}
}
}
}
builder.WriteString("\n---\nGenerated by [Nuclei](https://github.com/projectdiscovery/nuclei)")
data := builder.String()
return data
+15
View File
@@ -7,6 +7,7 @@ import (
"github.com/projectdiscovery/nuclei/v2/pkg/output"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/dedupe"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/exporters/disk"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/exporters/sarif"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/trackers/github"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/trackers/gitlab"
"github.com/projectdiscovery/nuclei/v2/pkg/reporting/trackers/jira"
@@ -28,6 +29,8 @@ type Options struct {
Jira *jira.Options `yaml:"jira"`
// DiskExporter contains configuration options for Disk Exporter Module
DiskExporter *disk.Options `yaml:"disk"`
// SarifExporter contains configuration options for Sarif Exporter Module
SarifExporter *sarif.Options `yaml:"sarif"`
}
// Filter filters the received event and decides whether to perform
@@ -79,6 +82,8 @@ type Tracker interface {
// Exporter is an interface implemented by an issue exporter
type Exporter interface {
// Close closes the exporter after operation
Close() error
// Export exports an issue to an exporter
Export(event *output.ResultEvent) error
}
@@ -129,6 +134,13 @@ func New(options *Options, db string) (*Client, error) {
}
client.exporters = append(client.exporters, exporter)
}
if options.SarifExporter != nil {
exporter, err := sarif.New(options.SarifExporter)
if err != nil {
return nil, errors.Wrap(err, "could not create exporting client")
}
client.exporters = append(client.exporters, exporter)
}
storage, err := dedupe.New(db)
if err != nil {
return nil, err
@@ -140,6 +152,9 @@ func New(options *Options, db string) (*Client, error) {
// Close closes the issue tracker reporting client
func (c *Client) Close() {
c.dedupe.Close()
for _, exporter := range c.exporters {
exporter.Close()
}
}
// CreateIssue creates an issue in the tracker
+72 -7
View File
@@ -20,6 +20,8 @@ type Integration struct {
// Options contains the configuration options for jira client
type Options struct {
// Cloud value is set to true when Jira cloud is used
Cloud bool `yaml:"cloud"`
// URL is the URL of the jira server
URL string `yaml:"url"`
// AccountID is the accountID of the jira user.
@@ -36,8 +38,12 @@ type Options struct {
// New creates a new issue tracker integration client based on options.
func New(options *Options) (*Integration, error) {
username := options.Email
if !options.Cloud {
username = options.AccountID
}
tp := jira.BasicAuthTransport{
Username: options.Email,
Username: username,
Password: options.Token,
}
jiraClient, err := jira.NewClient(tp.Client(), options.URL)
@@ -51,15 +57,27 @@ func New(options *Options) (*Integration, error) {
func (i *Integration) CreateIssue(event *output.ResultEvent) error {
summary := format.Summary(event)
issueData := &jira.Issue{
Fields: &jira.IssueFields{
Assignee: &jira.User{AccountID: i.options.AccountID},
Reporter: &jira.User{AccountID: i.options.AccountID},
fields := &jira.IssueFields{
Assignee: &jira.User{AccountID: i.options.AccountID},
Reporter: &jira.User{AccountID: i.options.AccountID},
Description: jiraFormatDescription(event),
Type: jira.IssueType{Name: i.options.IssueType},
Project: jira.Project{Key: i.options.ProjectName},
Summary: summary,
}
// On-prem version of Jira server does not use AccountID
if !i.options.Cloud {
fields = &jira.IssueFields{
Assignee: &jira.User{Name: i.options.AccountID},
Description: jiraFormatDescription(event),
Type: jira.IssueType{Name: i.options.IssueType},
Project: jira.Project{Key: i.options.ProjectName},
Summary: summary,
},
}
}
issueData := &jira.Issue{
Fields: fields,
}
_, resp, err := i.jira.Issue.Create(issueData)
if err != nil {
@@ -92,6 +110,9 @@ func jiraFormatDescription(event *output.ResultEvent) string {
builder.WriteString(event.Timestamp.Format("Mon Jan 2 15:04:05 -0700 MST 2006"))
builder.WriteString("\n\n*Template Information*\n\n| Key | Value |\n")
for k, v := range event.Info {
if k == "reference" {
continue
}
builder.WriteString(fmt.Sprintf("| %s | %s |\n", k, v))
}
builder.WriteString("\n*Request*\n\n{code}\n")
@@ -107,7 +128,7 @@ func jiraFormatDescription(event *output.ResultEvent) string {
builder.WriteString("\n{code}\n\n")
if len(event.ExtractedResults) > 0 || len(event.Metadata) > 0 {
builder.WriteString("*Extra Information*\n\n")
builder.WriteString("\n*Extra Information*\n\n")
if len(event.ExtractedResults) > 0 {
builder.WriteString("*Extracted results*:\n\n")
for _, v := range event.ExtractedResults {
@@ -129,6 +150,50 @@ func jiraFormatDescription(event *output.ResultEvent) string {
builder.WriteString("\n")
}
}
if event.Interaction != nil {
builder.WriteString("*Interaction Data*\n---\n")
builder.WriteString(event.Interaction.Protocol)
if event.Interaction.QType != "" {
builder.WriteString(" (")
builder.WriteString(event.Interaction.QType)
builder.WriteString(")")
}
builder.WriteString(" Interaction from ")
builder.WriteString(event.Interaction.RemoteAddress)
builder.WriteString(" at ")
builder.WriteString(event.Interaction.UniqueID)
if event.Interaction.RawRequest != "" {
builder.WriteString("\n\n*Interaction Request*\n\n{code}\n")
builder.WriteString(event.Interaction.RawRequest)
builder.WriteString("\n{code}\n")
}
if event.Interaction.RawResponse != "" {
builder.WriteString("\n*Interaction Response*\n\n{code}\n")
builder.WriteString(event.Interaction.RawResponse)
builder.WriteString("\n{code}\n")
}
}
if d, ok := event.Info["reference"]; ok {
builder.WriteString("\nReference: \n")
switch v := d.(type) {
case string:
if !strings.HasPrefix(v, "-") {
builder.WriteString("- ")
}
builder.WriteString(v)
case []interface{}:
slice := types.ToStringSlice(v)
for i, item := range slice {
builder.WriteString("- ")
builder.WriteString(item)
if len(slice)-1 != i {
builder.WriteString("\n")
}
}
}
}
builder.WriteString("\n---\nGenerated by [Nuclei|https://github.com/projectdiscovery/nuclei]")
data := builder.String()
return data
+1
View File
@@ -142,6 +142,7 @@ func Parse(filePath string, options protocols.ExecuterOptions) (*Template, error
if template.Executer == nil && template.CompiledWorkflow == nil {
return nil, errors.New("cannot create template executer")
}
template.Path = filePath
return template, nil
}
+10 -8
View File
@@ -17,22 +17,24 @@ type Template struct {
// Info contains information about the template
Info map[string]interface{} `yaml:"info"`
// RequestsHTTP contains the http request to make in the template
RequestsHTTP []*http.Request `yaml:"requests,omitempty"`
RequestsHTTP []*http.Request `yaml:"requests,omitempty" json:"requests"`
// RequestsDNS contains the dns request to make in the template
RequestsDNS []*dns.Request `yaml:"dns,omitempty"`
RequestsDNS []*dns.Request `yaml:"dns,omitempty" json:"dns"`
// RequestsFile contains the file request to make in the template
RequestsFile []*file.Request `yaml:"file,omitempty"`
RequestsFile []*file.Request `yaml:"file,omitempty" json:"file"`
// RequestsNetwork contains the network request to make in the template
RequestsNetwork []*network.Request `yaml:"network,omitempty"`
RequestsNetwork []*network.Request `yaml:"network,omitempty" json:"network"`
// RequestsHeadless contains the headless request to make in the template.
RequestsHeadless []*headless.Request `yaml:"headless,omitempty"`
RequestsHeadless []*headless.Request `yaml:"headless,omitempty" json:"headless"`
// Workflows is a yaml based workflow declaration code.
workflows.Workflow `yaml:",inline,omitempty"`
CompiledWorkflow *workflows.Workflow `yaml:"-"`
CompiledWorkflow *workflows.Workflow `yaml:"-" json:"-" jsonschema:"-"`
// TotalRequests is the total number of requests for the template.
TotalRequests int `yaml:"-"`
TotalRequests int `yaml:"-" json:"-"`
// Executer is the actual template executor for running template requests
Executer protocols.Executer `yaml:"-"`
Executer protocols.Executer `yaml:"-" json:"-"`
Path string `yaml:"-" json:"-"`
}
+16 -2
View File
@@ -25,6 +25,8 @@ type Options struct {
BurpCollaboratorBiid string
// ProjectPath allows nuclei to use a user defined project folder
ProjectPath string
// InteractshURL is the URL for the interactsh server.
InteractshURL string
// Target is a single URL/Domain to scan using a template
Target string
// Targets specifies the targets to scan using templates.
@@ -45,6 +47,8 @@ type Options struct {
ReportingConfig string
// DiskExportDirectory is the directory to export reports in markdown on disk to
DiskExportDirectory string
// SarifExport is the file to export sarif output format to
SarifExport string
// ResolversFile is a file containing resolvers for nuclei.
ResolversFile string
// StatsInterval is the number of seconds to display stats after
@@ -63,6 +67,16 @@ type Options struct {
RateLimit int
// PageTimeout is the maximum time to wait for a page in seconds
PageTimeout int
// InteractionsCacheSize is the number of interaction-url->req to keep in cache at a time.
InteractionsCacheSize int
// InteractionsPollDuration is the number of seconds to wait before each interaction poll
InteractionsPollDuration int
// Eviction is the number of seconds after which to automatically discard
// interaction requests.
InteractionsEviction int
// InteractionsColldownPeriod is additional seconds to wait for interactions after closing
// of the poller.
InteractionsColldownPeriod int
// OfflineHTTP is a flag that specific offline processing of http response
// using same matchers/extractors from http protocol without the need
// to send a new request, reading responses from a file.
@@ -73,8 +87,6 @@ type Options struct {
ShowBrowser bool
// SytemResolvers enables override of nuclei's DNS client opting to use system resolver stack.
SystemResolvers bool
// RandomAgent generates random User-Agent
RandomAgent bool
// Metrics enables display of metrics via an http endpoint
Metrics bool
// Debug mode allows debugging request/responses for the engine
@@ -113,4 +125,6 @@ type Options struct {
Project bool
// NewTemplates only runs newly added templates from the repository
NewTemplates bool
// NoInteractsh disables use of interactsh server for interaction polling
NoInteractsh bool
}