Compare commits

...
Author SHA1 Message Date
Ice3man 4626b56dd0 bugfix: fixed issues related to linear output + misc 2024-09-13 23:39:39 +05:30
Ice3man d946a7890a feat: fixed unmarshal issues 2024-09-05 15:46:11 +05:30
Ice3man e941b685e0 misc 2024-09-05 01:06:52 +05:30
Ice3man 55639e32c8 feat: added linear issue tracker support to nuclei 2024-09-05 01:05:24 +05:30
sandeep 042b33de3d version update 2024-09-02 17:38:48 +05:30
dependabot[bot] 5bb730849c chore(deps): bump github.com/projectdiscovery/httpx from 1.6.7 to 1.6.8 (#5590)
Bumps [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) from 1.6.7 to 1.6.8.
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Changelog](https://github.com/projectdiscovery/httpx/blob/main/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.6.7...v1.6.8)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/httpx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-02 15:55:34 +05:30
dependabot[bot] 7bc257d7a5 chore(deps): bump github.com/projectdiscovery/ratelimit (#5588)
Bumps [github.com/projectdiscovery/ratelimit](https://github.com/projectdiscovery/ratelimit) from 0.0.49 to 0.0.53.
- [Release notes](https://github.com/projectdiscovery/ratelimit/releases)
- [Commits](https://github.com/projectdiscovery/ratelimit/compare/v0.0.49...v0.0.53)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/ratelimit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-02 15:47:35 +05:30
dependabot[bot] 04a6159713 chore(deps): bump github.com/projectdiscovery/retryablehttp-go (#5591)
Bumps [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) from 1.0.74 to 1.0.76.
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.0.74...v1.0.76)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-02 15:46:16 +05:30
dependabot[bot] 0c4c0af4e9 chore(deps): bump github.com/projectdiscovery/dsl from 0.1.8 to 0.2.1 (#5587)
Bumps [github.com/projectdiscovery/dsl](https://github.com/projectdiscovery/dsl) from 0.1.8 to 0.2.1.
- [Release notes](https://github.com/projectdiscovery/dsl/releases)
- [Commits](https://github.com/projectdiscovery/dsl/compare/v0.1.8...v0.2.1)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/dsl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-02 15:36:39 +05:30
ghost cb05f55251 Auto Generate Syntax Docs + JSONSchema [Mon Sep 2 10:01:02 UTC 2024] 🤖 2024-09-02 10:01:02 +00:00
Dwi Siswanto 841d8913e5 feat(headless): add ActionWaitDialog type (#5545)
* feat(headless): add `dialog` action type

also implement it

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor(headless): add `ActionData` for action output datas

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor(headless): rm `value` arg for `*Page.HandleDialog`

also:
* expose `err` from \*proto.PageHandleJavaScriptDialog`
* conditional ActionData assignment based on

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor(headless): rename to `ActionWaitDialog`

Signed-off-by: Dwi Siswanto <[email protected]>

* test(headless): fix mismatch assertion of `src` output of `ActionGetResource`

Signed-off-by: Dwi Siswanto <[email protected]>

* test(headless): add TestActionWaitDialog test case

Signed-off-by: Dwi Siswanto <[email protected]>

* feat(headless): add `GetActionDataWithDefault` generic func

Signed-off-by: Dwi Siswanto <[email protected]>

* feat(headless): implement `GetActionDataWithDefault`

to `header` & `status_code`

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor(headless): use `mapsutil.Map` instead

Signed-off-by: Dwi Siswanto <[email protected]>

* Revert "feat(headless): add `GetActionDataWithDefault` generic func"

This reverts commit fa12e0d6a2.

---------

Signed-off-by: Dwi Siswanto <[email protected]>
2024-09-02 15:29:52 +05:30
dependabot[bot] 9a5272985c Merge pull request #5589 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/fastdialer-0.2.7 2024-09-02 05:38:41 +00:00
dependabot[bot] fdb8e0a8c6 chore(deps): bump github.com/projectdiscovery/fastdialer
Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.2.5 to 0.2.7.
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.2.5...v0.2.7)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-09-02 05:11:44 +00:00
linchizhen 88b281ae47 chore: fix some function names in comment (#5586)
Signed-off-by: linchizhen <[email protected]>
2024-09-01 14:11:01 +05:30
Dwi Siswanto e0b2542868 feat: conditionally panic-recover (#5553)
* feat: conditionally panic-recover

As discussed with @Mzack9999, we should avoid
overusing panic-recover. We need to review the RCA
first to determine whether this is an exceptional
situation or if it's a higher-level function meant
to recover from a panic. This approach will help
us establish a robust error-handling strategy.

The implementation of panic-recover should be
conditional and NOT applied when running in a CI
environment AND IS temporary. Once we've caught
all errors and made the necessary corrections, we
can remove the deferred recover function.

Signed-off-by: Dwi Siswanto <[email protected]>

* chore(deps): bump `go-ci` to v1.0.2

Signed-off-by: Dwi Siswanto <[email protected]>

* chore(make): add `-race` to `GOFLAGS` in `test`

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-28 17:57:45 +05:30
Dogan Can Bakir 6b71af448a Fixed issue with -ms option to scan non accessible host (#5576)
* fail if OnResult callback is not called

* generate error message from error logs

* try..parse..

* fix lint

* add error message to last matcher event

* fix network protocol error logging

* log returned log from ExecuteWithResults

* add back specific logging

* clean up the msg

* minor

* init integration test for -ms

* add tests for http,network,js,ws protocols

* fix lint

* fix network test

* return err for dns protocol

* add integration test for dns protocol
2024-08-28 16:27:43 +05:30
Peter Dave Hello bf58b4d756 Remove conflicting -U option for apk in Dockerfile (#5578)
The `--no-cache` option updates the index and avoids temporary files, making `-U` unnecessary, consistent with the subsequent `apk add`.
2024-08-28 16:24:15 +05:30
Dwi Siswanto aac1af1308 refactor(fuzz): use mapsutil.Map type (#5558)
* refactor(fuzz): use `mapsutil.Map` type

Signed-off-by: Dwi Siswanto <[email protected]>

* fix(headless): handle empty `key` in `*Value.SetParsedValue`

Signed-off-by: Dwi Siswanto <[email protected]>

* feat(fuzz): add type assertion checks

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-28 12:41:02 +05:30
Dogan Can Bakir 4a85e73fbb skip dependabot prs (#5575) 2024-08-26 22:18:45 +05:30
dependabot[bot] c23e1864d6 Merge pull request #5574 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/tlsx-1.1.7 2024-08-26 06:21:12 +00:00
dependabot[bot] 2670b6865f chore(deps): bump github.com/projectdiscovery/tlsx from 1.1.6 to 1.1.7
Bumps [github.com/projectdiscovery/tlsx](https://github.com/projectdiscovery/tlsx) from 1.1.6 to 1.1.7.
- [Release notes](https://github.com/projectdiscovery/tlsx/releases)
- [Changelog](https://github.com/projectdiscovery/tlsx/blob/main/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/tlsx/compare/v1.1.6...v1.1.7)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/tlsx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-26 05:53:48 +00:00
dependabot[bot] b424ba9f50 Merge pull request #5570 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/goflags-0.1.64 2024-08-26 05:53:42 +00:00
dependabot[bot] faed7798fc Merge pull request #5571 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/utils-0.2.7 2024-08-26 05:52:06 +00:00
dependabot[bot] 253fbc3619 chore(deps): bump github.com/projectdiscovery/goflags
Bumps [github.com/projectdiscovery/goflags](https://github.com/projectdiscovery/goflags) from 0.1.63 to 0.1.64.
- [Release notes](https://github.com/projectdiscovery/goflags/releases)
- [Commits](https://github.com/projectdiscovery/goflags/compare/v0.1.63...v0.1.64)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/goflags
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-26 05:26:23 +00:00
dependabot[bot] 4a39fdb8d6 chore(deps): bump github.com/projectdiscovery/utils from 0.2.5 to 0.2.7
Bumps [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils) from 0.2.5 to 0.2.7.
- [Release notes](https://github.com/projectdiscovery/utils/releases)
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md)
- [Commits](https://github.com/projectdiscovery/utils/compare/v0.2.5...v0.2.7)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/utils
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-26 05:26:22 +00:00
dependabot[bot] 5627da8c19 Merge pull request #5573 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/gologger-1.1.21 2024-08-26 05:25:22 +00:00
dependabot[bot] 710f1ad240 chore(deps): bump github.com/projectdiscovery/gologger
Bumps [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) from 1.1.19 to 1.1.21.
- [Release notes](https://github.com/projectdiscovery/gologger/releases)
- [Commits](https://github.com/projectdiscovery/gologger/compare/v1.1.19...v1.1.21)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/gologger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-26 05:02:29 +00:00
Dogan Can Bakir 56af26a502 Add auto assign (#5555)
* add auto assign

* commit to last commit
2024-08-22 18:19:07 +05:30
Mzack9999 8e1072e2b0 Merge pull request #5551 from AdallomRoy/dev
Upgrade gitlab
2024-08-21 17:06:55 +02:00
Mzack9999 b53b530b6a Merge pull request #5547 from projectdiscovery/fix_race_condition
fix race condition
2024-08-21 17:03:21 +02:00
mzack9999 bfdc507c6c lint 2024-08-21 16:34:53 +02:00
mzack9999 ddf9b2fbec fix 2024-08-21 16:29:00 +02:00
mzack9999 5e102b782b fixing race + nil crash 2024-08-21 16:09:47 +02:00
Doğan Can Bakır d1f4c98cd7 Revert "remove redundant code"
This reverts commit 35a0d673ad.
2024-08-21 15:03:41 +03:00
Roy Reznik 87c8cd8e01 Upgrade gitlab 2024-08-21 11:44:01 +01:00
Doğan Can Bakır 35a0d673ad remove redundant code 2024-08-21 11:36:33 +03:00
Doğan Can Bakır 46782ff90c use sync.Once 2024-08-21 11:26:17 +03:00
Dwi Siswanto b1152efd42 chore(make): make it DRY (#5549)
Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-20 13:36:39 +05:30
Dwi Siswanto d84aea93cc chore: sorts gitignore (#5548)
Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-20 13:34:01 +05:30
Doğan Can Bakır 50023428d0 fix lint 2024-08-19 23:09:07 +03:00
Doğan Can Bakır 7af08e2b04 Merge branch 'dev' into fix_race_condition 2024-08-19 23:06:53 +03:00
Doğan Can Bakır 3064788d35 fix race condition 2024-08-19 23:02:27 +03:00
0da993afe6 Merge commit from fork
* fix template signature verification

* fix signature pattern check

* add tests

* remove signature count constraint, check for lines len

* Add more tests

* Centralize signature extraction logic in signer package

* Move signature handling in Sign function to beginning

* Remove comment

* Revert `NewTemplateSigVerifier`

* update tests

* use ExtractSignatureAndContent func

* Allow signing code templates

* Remove unused const

---------

Co-authored-by: Doğan Can Bakır <[email protected]>
Co-authored-by: Guy Goldenberg <[email protected]>
2024-08-19 18:02:54 +05:30
dependabot[bot] 4d5eb9c484 Merge pull request #5543 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/retryabledns-1.0.72 2024-08-19 07:24:14 +00:00
dependabot[bot] 8a3818230d chore(deps): bump github.com/projectdiscovery/retryabledns
Bumps [github.com/projectdiscovery/retryabledns](https://github.com/projectdiscovery/retryabledns) from 1.0.69 to 1.0.72.
- [Release notes](https://github.com/projectdiscovery/retryabledns/releases)
- [Commits](https://github.com/projectdiscovery/retryabledns/compare/v1.0.69...v1.0.72)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/retryabledns
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-19 06:59:19 +00:00
dependabot[bot] 943a6f6643 Merge pull request #5541 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/retryablehttp-go-1.0.74 2024-08-19 06:57:53 +00:00
dependabot[bot] 091ca32b1c chore(deps): bump github.com/projectdiscovery/retryablehttp-go
Bumps [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) from 1.0.71 to 1.0.74.
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.0.71...v1.0.74)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-19 06:33:12 +00:00
dependabot[bot] 326f8774fd Merge pull request #5544 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/fastdialer-0.2.5 2024-08-19 06:32:03 +00:00
dependabot[bot] c94d226a49 chore(deps): bump github.com/projectdiscovery/fastdialer
Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.2.3 to 0.2.5.
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.2.3...v0.2.5)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-19 06:07:14 +00:00
dependabot[bot] c9fe07557c Merge pull request #5542 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/hmap-0.0.55 2024-08-19 06:06:13 +00:00
dependabot[bot] e16350cde6 chore(deps): bump github.com/projectdiscovery/hmap from 0.0.54 to 0.0.55
Bumps [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) from 0.0.54 to 0.0.55.
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.54...v0.0.55)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/hmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-19 05:39:12 +00:00
dependabot[bot] 0f79044dae Merge pull request #5540 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/clistats-0.1.0 2024-08-19 05:37:55 +00:00
dependabot[bot] c6bafd88eb chore(deps): bump github.com/projectdiscovery/clistats
Bumps [github.com/projectdiscovery/clistats](https://github.com/projectdiscovery/clistats) from 0.0.20 to 0.1.0.
- [Release notes](https://github.com/projectdiscovery/clistats/releases)
- [Commits](https://github.com/projectdiscovery/clistats/compare/v0.0.20...v0.1.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/clistats
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-19 05:12:33 +00:00
Ice3manandsandeep 80b3cc0627 feat: jira accept issue-type and project ids (#5537)
* feat: jira accept issue-type and project ids

* remove validation for project name

---------

Co-authored-by: sandeep <[email protected]>
2024-08-19 06:20:12 +05:30
Dwi Siswanto 02fadbcf77 chore: migrate issue template to issue form (#5538)
Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-19 04:28:30 +05:30
sandeep f2f250738c version update 2024-08-16 20:44:39 +05:30
Tarun KoyalwarandDwi Siswanto 1c76398aea lint error fixes (#5531)
* lint error fixes

* chore: satisfy non-constant format str in call lint (govet)

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
Co-authored-by: Dwi Siswanto <[email protected]>
2024-08-16 20:31:23 +05:30
Tryfon Papatriantafyllou 0675aa48a3 Circular References in OpenAPI 3.0 fixed (#5491)
* Circular References in OpenAPI 3.0 fixed

* Fixing Swagger_test
2024-08-16 18:34:47 +05:30
Ramana Reddy f29b94521e fix unresolved variables in dast templates (#5443)
* fix unresolved variables in dast templates

* dedupe interactsh urls

* misc update
2024-08-16 18:19:44 +05:30
ghost d20ec34f63 Auto Generate Syntax Docs + JSONSchema [Fri Aug 16 12:41:50 UTC 2024] 🤖 2024-08-16 12:41:50 +00:00
Dwi Siswanto 1af29f97a9 feat(http): add skip-secret-file field (#5522)
* feat(http): add `BypassSecretFile` field

Signed-off-by: Dwi Siswanto <[email protected]>

* feat(http): conditionally apply auth strategies

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor(http): rename `BypassSecretFile` field to `SkipSecretFile`

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-16 18:10:48 +05:30
sandeep 1293a34707 dep update 2024-08-16 18:03:53 +05:30
Tryfon Papatriantafyllou c6e5bdd857 Fixing the server URL path for OpenAPI scanning (#5504)
* fix_openAPI_serverURL_path

* Issue #5503
2024-08-16 17:37:02 +05:30
Ramana ReddyandTarun Koyalwar 2f7eea410d Add team-id option (#5523)
* add team-id option

* fix dashboard url when uploading to team

---------

Co-authored-by: Tarun Koyalwar <[email protected]>
2024-08-16 13:27:26 +05:30
Ramana Reddy 2609d2d135 feat: add support for multiple auth strategies per target from secrets file (#5500) 2024-08-16 11:59:15 +05:30
Dogan Can Bakir e0466e102c redact output (#5463)
* redact output

* update regex

* redact matchet-at
2024-08-16 11:42:38 +05:30
Dwi Siswanto 1e49cd9800 ci: don't clean modules cache (#5519)
* chore(make): add `verify` command

Signed-off-by: Dwi Siswanto <[email protected]>

* ci: remove hygiene (`clean -modcache`) with verify instead

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
2024-08-15 18:45:42 +05:30
dependabot[bot] a5c7fe6094 Merge pull request #5511 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/goflags-0.1.63 2024-08-15 12:25:47 +00:00
dependabot[bot] f363b307d4 chore(deps): bump github.com/projectdiscovery/goflags
Bumps [github.com/projectdiscovery/goflags](https://github.com/projectdiscovery/goflags) from 0.1.62 to 0.1.63.
- [Release notes](https://github.com/projectdiscovery/goflags/releases)
- [Commits](https://github.com/projectdiscovery/goflags/compare/v0.1.62...v0.1.63)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/goflags
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2024-08-12 06:06:05 +00:00
74 changed files with 1985 additions and 472 deletions

No files matched your search

+72
View File
@@ -0,0 +1,72 @@
name: Bug Report
description: Create a report to help us to improve the Nuclei.
title: "[BUG] ..."
labels: ["Type: Bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report!
For support requests, FAQs or "How to" questions, please use the [GitHub Discussions](https://github.com/projectdiscovery/nuclei/discussions) section instead or join our [Discord server](https://discord.gg/projectdiscovery) to discuss the idea on the **#nuclei** channel.
:warning: **Issues missing important information may be closed without further investigation.**
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the bug you encountered.
options:
- label: I have searched the existing issues.
required: true
- type: textarea
attributes:
label: Current Behavior
description: A concise description of what you're experiencing.
validations:
required: true
- type: textarea
attributes:
label: Expected Behavior
description: A concise description of what you expected to happen.
validations:
required: true
- type: textarea
attributes:
label: Steps To Reproduce
description: |
Steps to reproduce the behavior, for example, commands to run Nuclei.
:warning: **Please redact any literal target hosts/URLs or other sensitive information.**
placeholder: |
1. Run `nuclei -t ...`
validations:
required: true
- type: textarea
attributes:
label: Relevant log output
description: Please copy and paste any relevant log output. This will be automatically formatted into code, so no need for backticks.
render: shell
- type: textarea
attributes:
label: Environment
description: |
Examples:
- **OS**: Ubuntu 20.04
- **Nuclei** (`nuclei -version`): v3.3.1
- **Go** (`go version`): go1.22.0 _(only if you've installed it via the `go install` command)_
value: |
- OS:
- Nuclei:
- Go:
render: markdown
validations:
required: true
- type: textarea
attributes:
label: Anything else?
description: |
Links? References? Templates? Anything that will give us more context about the issue you are encountering!
Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
validations:
required: false
@@ -0,0 +1,37 @@
name: Feature Request
description: Request feature to implement in the Nuclei.
title: "[FEATURE] ..."
labels: ["Type: Enhancement"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this feature request!
Please make sure to provide a detailed description with all the relevant information that might be required to start working on this feature. In case you are not sure about your request or whether the particular feature is already supported or not, please [start a discussion](https://github.com/projectdiscovery/nuclei/discussions/categories/ideas) instead.
Join our [Discord server](https://discord.gg/projectdiscovery) to discuss the idea on the **#nuclei** channel.
- type: textarea
attributes:
label: Describe your feature request
description: A clear and concise description of feature to implement.
validations:
required: true
- type: textarea
attributes:
label: Describe the use case of the feature
description: A clear and concise description of the feature request's motivation and the use-cases in which it could be useful.
validations:
required: true
- type: textarea
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered.
validations:
required: false
- type: textarea
attributes:
label: Additional context
description: Add any other context about the feature request here.
validations:
required: false
-21
View File
@@ -1,21 +0,0 @@
---
name: Feature request
about: Request feature to implement in this project
title: ""
labels: 'Type: Enhancement'
assignees: ''
---
<!--
1. Please make sure to provide a detailed description with all the relevant information that might be required to start working on this feature.
2. In case you are not sure about your request or whether the particular feature is already supported or not, please start a discussion instead.
3. GitHub Discussion: https://github.com/projectdiscovery/nuclei/discussions/categories/ideas
4. Join our discord server at https://discord.gg/projectdiscovery to discuss the idea on the #nuclei channel.
-->
### Please describe your feature request:
<!-- A clear and concise description of feature to implement -->
### Describe the use case of this feature:
<!-- A clear and concise description of the feature request's motivation and the use-cases in which it could be useful. -->
-41
View File
@@ -1,41 +0,0 @@
---
name: Issue report
about: Create a report to help us to improve the project
labels: 'Type: Bug'
---
<!--
1. Please search to see if an issue already exists for the bug you encountered.
2. For support requests, FAQs or "How to" questions, please use the GitHub Discussions section instead - https://github.com/projectdiscovery/nuclei/discussions or
3. Join our discord server at https://discord.gg/projectdiscovery and post the question on the #nuclei channel.
-->
<!-- ISSUES MISSING IMPORTANT INFORMATION MAY BE CLOSED WITHOUT INVESTIGATION. -->
### Nuclei version:
<!-- You can find current version of nuclei with "nuclei -version" -->
<!-- We only accept issues that are reproducible on the latest version of nuclei. -->
<!-- You can find the latest version of project at https://github.com/projectdiscovery/nuclei/releases/ -->
## Nuclei command:
<!-- Provide the exact command you used to run Nuclei. -->
<!-- Please redact any literal target hosts/URLs or other sensitive information. -->
### Current Behavior:
<!-- A concise description of what you're experiencing. -->
### Expected Behavior:
<!-- A concise description of what you expected to happen. -->
### Steps To Reproduce:
<!--
Example: steps to reproduce the behavior:
1. Run 'nuclei -t ... -u ..'
2. See error...
-->
### Anything else:
<!-- Links? References? Screenshots? Anything that will give us more context about the issue that you are encountering! -->
+7
View File
@@ -0,0 +1,7 @@
addReviewers: true
reviewers:
- dogancanbakir
numberOfReviewers: 1
skipKeywords:
- '@dependabot'
+2 -4
View File
@@ -27,10 +27,8 @@ jobs:
with:
python-version: '3.10'
- name: Go Mod hygiene
run: |
go clean -modcache
go mod tidy
- name: Verify Go modules
run: make verify
- name: Build
run: go build .
+2 -4
View File
@@ -22,10 +22,8 @@ jobs:
- name: Set up Go
uses: projectdiscovery/actions/setup/go@v1
- name: Go Mod hygine
run: |
go clean -modcache
go mod tidy
- name: Verify Go modules
run: make verify
# Max GH exection time 6H => timeout after that
- name: Running performance with big list
+36 -37
View File
@@ -1,43 +1,42 @@
# Misc
**/*-cache
**/*-config
**/.cache
*.DS_Store
*.exe
.devcontainer
.gitignore
.idea
.vscode
.devcontainer
vendor
# Binaries
**/bindgen
**/debug-*
**/docgen
**/functional-test
**/fuzzplayground
**/integration-test
**/jsdocgen
**/main
**/memogen
**/nuclei
**/nuclei-stats*
**/nuclei_dev
**/nuclei_main
**/scan-charts
**/scrapefunc
**/scrapefuncs
**/tsgen
# Templates
/*.yaml
# Generated docs
/dsl.md
# Junk
dist
integration_tests/nuclei
integration_tests/.cache
integration_tests/integration-test
cmd/nuclei/main
cmd/nuclei/nuclei
cmd/integration-test/nuclei
cmd/functional-test/nuclei_dev
cmd/functional-test/nuclei_main
cmd/functional-test/functional-test
cmd/docgen/docgen
pkg/protocols/common/helpers/deserialization/testdata/Deserialize.class
pkg/protocols/common/helpers/deserialization/testdata/ValueObject.class
pkg/protocols/common/helpers/deserialization/testdata/ValueObject2.ser
*.exe
.gitignore
pkg/js/devtools/bindgen/cmd/bindgen/bindgen
pkg/js/devtools/jsdocgen/jsdocgen
pkg/js/devtools/tsgen/tsgen
pkg/js/devtools/tsgen/cmd/tsgen/tsgen
*.DS_Store
pkg/protocols/headless/engine/.cache
/nuclei
/bindgen
/jsdocgen
/tsgen
/scrapefuncs
/integration_tests/.cache/
/*.yaml
**/*-config
**/*-cache
/fuzzplayground
integration_tests/fuzzplayground
/dsl.md
/nuclei-stats
/nuclei-stats-*
/scan-charts
/**/debug-*
vendor
+1 -1
View File
@@ -9,7 +9,7 @@ RUN go build ./cmd/nuclei
# Release
FROM alpine:3.18.6
RUN apk -U upgrade --no-cache \
RUN apk upgrade --no-cache \
&& apk add --no-cache bind-tools chromium ca-certificates
COPY --from=build-env /app/nuclei /usr/local/bin/
+105 -39
View File
@@ -1,62 +1,128 @@
# Go parameters
GOCMD=go
GOBUILD=$(GOCMD) build
GOMOD=$(GOCMD) mod
GOTEST=$(GOCMD) test
GOCMD := go
GOBUILD := $(GOCMD) build
GOBUILD_OUTPUT :=
GOBUILD_PACKAGES :=
GOBUILD_ADDITIONAL_ARGS :=
GOMOD := $(GOCMD) mod
GOTEST := $(GOCMD) test
GOFLAGS := -v
# This should be disabled if the binary uses pprof
LDFLAGS := -s -w
ifneq ($(shell go env GOOS),darwin)
LDFLAGS := -extldflags "-static"
LDFLAGS = -extldflags "-static"
endif
.PHONY: all build build-stats scan-charts docs test integration functional tidy devtools jsupdate ts fuzzplayground memogen dsl-docs
.PHONY: all build build-stats clean devtools-all devtools-bindgen devtools-scrapefuncs
.PHONY: devtools-tsgen docs dsl-docs functional fuzzplayground go-build integration
.PHONY: jsupdate-all jsupdate-bindgen jsupdate-tsgen memogen scan-charts test tidy ts verify
all: build
build:
rm -f nuclei 2>/dev/null
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "nuclei" cmd/nuclei/main.go
build-stats:
rm -f nuclei-stats 2>/dev/null
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -tags=stats -o "nuclei-stats" cmd/nuclei/main.go
scan-charts:
rm -f scan-charts 2>/dev/null
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "scan-charts" cmd/scan-charts/main.go
clean:
rm -f '${GOBUILD_OUTPUT}' 2>/dev/null
go-build: clean
go-build:
$(GOBUILD) $(GOFLAGS) -ldflags '${LDFLAGS}' $(GOBUILD_ADDITIONAL_ARGS) \
-o '${GOBUILD_OUTPUT}' $(GOBUILD_PACKAGES)
build: GOBUILD_OUTPUT = nuclei
build: GOBUILD_PACKAGES = cmd/nuclei/main.go
build: go-build
build-stats: GOBUILD_OUTPUT = nuclei-stats
build-stats: GOBUILD_PACKAGES = cmd/nuclei/main.go
build-stats: GOBUILD_ADDITIONAL_ARGS = -tags=stats
build-stats: go-build
scan-charts: GOBUILD_OUTPUT = scan-charts
scan-charts: GOBUILD_PACKAGES = cmd/scan-charts/main.go
scan-charts: go-build
docs: GOBUILD_OUTPUT = cmd/docgen/docgen
docs: GOBUILD_PACKAGES = cmd/docgen/docgen.go
docs: bin = dstdocgen
docs:
if ! which dstdocgen > /dev/null; then
echo -e "Command not found! Install? (y/n) \c"
go get -v github.com/projectdiscovery/yamldoc-go/cmd/docgen/dstdocgen
@if ! which $(bin) >/dev/null; then \
read -p "${bin} not found. Do you want to install it? (y/n) " answer; \
if [ "$$answer" = "y" ]; then \
echo "Installing ${bin}..."; \
go get -v github.com/projectdiscovery/yamldoc-go/cmd/docgen/$(bin); \
go install -v github.com/projectdiscovery/yamldoc-go/cmd/docgen/$(bin); \
else \
echo "Please install ${bin} manually."; \
exit 1; \
fi \
fi
# TODO: Handle the panic, so that we just need to run `go install $(bin)@latest` (line 51-52)
$(GOCMD) generate pkg/templates/templates.go
$(GOBUILD) -o "cmd/docgen/docgen" cmd/docgen/docgen.go
./cmd/docgen/docgen docs.md nuclei-jsonschema.json
$(GOBUILD) -o "${GOBUILD_OUTPUT}" $(GOBUILD_PACKAGES)
./$(GOBUILD_OUTPUT) docs.md nuclei-jsonschema.json
git reset --hard # line 59
test: GOFLAGS = -race -v
test:
$(GOTEST) $(GOFLAGS) ./...
integration:
cd integration_tests; bash run.sh
functional:
cd cmd/functional-test; bash run.sh
tidy:
$(GOMOD) tidy
devtools:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "bindgen" pkg/js/devtools/bindgen/cmd/bindgen/main.go
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "tsgen" pkg/js/devtools/tsgen/cmd/tsgen/main.go
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "scrapefuncs" pkg/js/devtools/scrapefuncs/main.go
jsupdate:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "bindgen" pkg/js/devtools/bindgen/cmd/bindgen/main.go
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "tsgen" pkg/js/devtools/tsgen/cmd/tsgen/main.go
./bindgen -dir pkg/js/libs -out pkg/js/generated
./tsgen -dir pkg/js/libs -out pkg/js/generated/ts
ts:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "tsgen" pkg/js/devtools/tsgen/cmd/tsgen/main.go
./tsgen -dir pkg/js/libs -out pkg/js/generated/ts
fuzzplayground:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "fuzzplayground" cmd/tools/fuzzplayground/main.go
verify: tidy
$(GOMOD) verify
devtools-bindgen: GOBUILD_OUTPUT = bindgen
devtools-bindgen: GOBUILD_PACKAGES = pkg/js/devtools/bindgen/cmd/bindgen/main.go
devtools-bindgen: go-build
devtools-tsgen: GOBUILD_OUTPUT = tsgen
devtools-tsgen: GOBUILD_PACKAGES = pkg/js/devtools/tsgen/cmd/tsgen/main.go
devtools-tsgen: go-build
devtools-scrapefuncs: GOBUILD_OUTPUT = scrapefuncs
devtools-scrapefuncs: GOBUILD_PACKAGES = pkg/js/devtools/scrapefuncs/main.go
devtools-scrapefuncs: go-build
devtools-all: devtools-bindgen devtools-tsgen devtools-scrapefuncs
jsupdate-bindgen: GOBUILD_OUTPUT = bindgen
jsupdate-bindgen: GOBUILD_PACKAGES = pkg/js/devtools/bindgen/cmd/bindgen/main.go
jsupdate-bindgen: go-build
jsupdate-bindgen:
./$(GOBUILD_OUTPUT) -dir pkg/js/libs -out pkg/js/generated
jsupdate-tsgen: GOBUILD_OUTPUT = tsgen
jsupdate-tsgen: GOBUILD_PACKAGES = pkg/js/devtools/tsgen/cmd/tsgen/main.go
jsupdate-tsgen: go-build
jsupdate-tsgen:
./$(GOBUILD_OUTPUT) -dir pkg/js/libs -out pkg/js/generated/ts
jsupdate-all: jsupdate-bindgen jsupdate-tsgen
ts: jsupdate-tsgen
fuzzplayground: GOBUILD_OUTPUT = fuzzplayground
fuzzplayground: GOBUILD_PACKAGES = cmd/tools/fuzzplayground/main.go
fuzzplayground: LDFLAGS = -s -w
fuzzplayground: go-build
memogen: GOBUILD_OUTPUT = memogen
memogen: GOBUILD_PACKAGES = cmd/memogen/memogen.go
memogen: go-build
memogen:
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "memogen" cmd/memogen/memogen.go
./memogen -src pkg/js/libs -tpl cmd/memogen/function.tpl
./$(GOBUILD_OUTPUT) -src pkg/js/libs -tpl cmd/memogen/function.tpl
dsl-docs: GOBUILD_OUTPUT = scrapefuncs
dsl-docs: GOBUILD_PACKAGES = pkg/js/devtools/scrapefuncs/main.go
dsl-docs:
rm -f dsl.md scrapefuncs 2>/dev/null
$(GOBUILD) $(GOFLAGS) -ldflags '$(LDFLAGS)' -o "scrapefuncs" pkg/js/devtools/scrapefuncs/main.go
./scrapefuncs -out dsl.md
./$(GOBUILD_OUTPUT) -out dsl.md
+2
View File
@@ -179,6 +179,7 @@ OUTPUT:
-se, -sarif-export string file to export results in SARIF format
-je, -json-export string file to export results in JSON format
-jle, -jsonl-export string file to export results in JSONL(ine) format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -310,6 +311,7 @@ STATISTICS:
CLOUD:
-auth configure projectdiscovery cloud (pdcp) api key (default true)
-tid, -team-id string upload scan results to given team id (optional) (default "none")
-cup, -cloud-upload upload scan results to pdcp dashboard
-sid, -scan-id string upload scan results to existing scan id (optional)
-sname, -scan-name string scan name to set (optional)
+15
View File
@@ -1404,6 +1404,19 @@ Valid values:
<div class="dd">
<code>skip-secret-file</code> <i>bool</i>
</div>
<div class="dt">
SkipSecretFile skips the authentication or authorization configured in the secret file.
</div>
<hr />
<div class="dd">
<code>cookie-reuse</code> <i>bool</i>
</div>
@@ -3669,6 +3682,8 @@ Enum Values:
- <code>waitevent</code>
- <code>dialog</code>
- <code>keyboard</code>
- <code>debug</code>
+1
View File
@@ -55,6 +55,7 @@ var (
"dsl": dslTestcases,
"flow": flowTestcases,
"javascript": jsTestcases,
"matcher-status": matcherStatusTestcases,
}
// flakyTests are run with a retry count of 3
flakyTests = map[string]bool{
+119
View File
@@ -0,0 +1,119 @@
package main
import (
"encoding/json"
"fmt"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/testutils"
)
var matcherStatusTestcases = []TestCaseInfo{
{Path: "protocols/http/get.yaml", TestCase: &httpNoAccess{}},
{Path: "protocols/network/net-https.yaml", TestCase: &networkNoAccess{}},
{Path: "protocols/headless/headless-basic.yaml", TestCase: &headlessNoAccess{}},
{Path: "protocols/javascript/net-https.yaml", TestCase: &javascriptNoAccess{}},
{Path: "protocols/websocket/basic.yaml", TestCase: &websocketNoAccess{}},
{Path: "protocols/dns/a.yaml", TestCase: &dnsNoAccess{}},
}
type httpNoAccess struct{}
func (h *httpNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "trust_me_bro.real", debug, "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error != "no address found for host" {
return fmt.Errorf("unexpected result: expecting \"no address found for host\" error but got none")
}
return nil
}
type networkNoAccess struct{}
// Execute executes a test case and returns an error if occurred
func (h *networkNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "trust_me_bro.real", debug, "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error != "no address found for host" {
return fmt.Errorf("unexpected result: expecting \"no address found for host\" error but got \"%s\"", event.Error)
}
return nil
}
type headlessNoAccess struct{}
// Execute executes a test case and returns an error if occurred
func (h *headlessNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "trust_me_bro.real", debug, "-headless", "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error == "" {
return fmt.Errorf("unexpected result: expecting an error but got \"%s\"", event.Error)
}
return nil
}
type javascriptNoAccess struct{}
// Execute executes a test case and returns an error if occurred
func (h *javascriptNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "trust_me_bro.real", debug, "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error == "" {
return fmt.Errorf("unexpected result: expecting an error but got \"%s\"", event.Error)
}
return nil
}
type websocketNoAccess struct{}
// Execute executes a test case and returns an error if occurred
func (h *websocketNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "ws://trust_me_bro.real", debug, "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error == "" {
return fmt.Errorf("unexpected result: expecting an error but got \"%s\"", event.Error)
}
return nil
}
type dnsNoAccess struct{}
// Execute executes a test case and returns an error if occurred
func (h *dnsNoAccess) Execute(filePath string) error {
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, "trust_me_bro.real", debug, "-ms", "-j")
if err != nil {
return err
}
event := &output.ResultEvent{}
_ = json.Unmarshal([]byte(results[0]), event)
if event.Error == "" {
return fmt.Errorf("unexpected result: expecting an error but got \"%s\"", event.Error)
}
return nil
}
+3
View File
@@ -12,6 +12,7 @@ import (
"strings"
"time"
_pdcp "github.com/projectdiscovery/nuclei/v3/internal/pdcp"
"github.com/projectdiscovery/utils/auth/pdcp"
"github.com/projectdiscovery/utils/env"
_ "github.com/projectdiscovery/utils/pprof"
@@ -272,6 +273,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringVarP(&options.SarifExport, "sarif-export", "se", "", "file to export results in SARIF format"),
flagSet.StringVarP(&options.JSONExport, "json-export", "je", "", "file to export results in JSON format"),
flagSet.StringVarP(&options.JSONLExport, "jsonl-export", "jle", "", "file to export results in JSONL(ine) format"),
flagSet.StringSliceVarP(&options.Redact, "redact", "rd", nil, "redact given list of keys from query parameter, request header and body", goflags.CommaSeparatedStringSliceOptions),
)
flagSet.CreateGroup("configs", "Configurations",
@@ -417,6 +419,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.CreateGroup("cloud", "Cloud",
flagSet.DynamicVar(&pdcpauth, "auth", "true", "configure projectdiscovery cloud (pdcp) api key"),
flagSet.StringVarP(&options.TeamID, "team-id", "tid", _pdcp.TeamIDEnv, "upload scan results to given team id (optional)"),
flagSet.BoolVarP(&options.EnableCloudUpload, "cloud-upload", "cup", false, "upload scan results to pdcp dashboard"),
flagSet.StringVarP(&options.ScanID, "scan-id", "sid", "", "upload scan results to existing scan id (optional)"),
flagSet.StringVarP(&options.ScanName, "scan-name", "sname", "", "scan name to set (optional)"),
+5 -5
View File
@@ -162,7 +162,7 @@ func process(opts options) error {
var updated bool // if max-requests is updated
dataString, updated, err = parseAndAddMaxRequests(templateCatalog, path, dataString)
if err != nil {
gologger.Info().Label("max-request").Msgf(logErrMsg(path, err, opts.debug, errFile))
gologger.Info().Label("max-request").Msg(logErrMsg(path, err, opts.debug, errFile))
} else {
if updated {
gologger.Info().Label("max-request").Msgf("✅ updated template: %s\n", path)
@@ -255,7 +255,7 @@ func enhanceTemplate(data string) (string, bool, error) {
return data, false, errorutil.New("validation failed").WithTag("validate")
}
if templateResp.Error.Name != "" {
return data, false, errorutil.New(templateResp.Error.Name)
return data, false, errorutil.New("%s", templateResp.Error.Name)
}
if strings.TrimSpace(templateResp.Enhanced) == "" && !templateResp.Lint {
if templateResp.LintError.Reason != "" {
@@ -289,7 +289,7 @@ func formatTemplate(data string) (string, bool, error) {
return data, false, errorutil.New("validation failed").WithTag("validate")
}
if templateResp.Error.Name != "" {
return data, false, errorutil.New(templateResp.Error.Name)
return data, false, errorutil.New("%s", templateResp.Error.Name)
}
if strings.TrimSpace(templateResp.Updated) == "" && !templateResp.Lint {
if templateResp.LintError.Reason != "" {
@@ -300,7 +300,7 @@ func formatTemplate(data string) (string, bool, error) {
return data, false, errorutil.New("template format failed")
}
// lintTemplateData lints template data using templateman lint api
// lintTemplate lints template data using templateman lint api
func lintTemplate(data string) (bool, error) {
resp, err := retryablehttp.DefaultClient().Post(fmt.Sprintf("%s/lint", tmBaseUrl), "application/x-yaml", strings.NewReader(data))
if err != nil {
@@ -345,7 +345,7 @@ func validateTemplate(data string) (bool, error) {
return false, errorutil.New("validation failed").WithTag("validate")
}
if validateResp.Error.Name != "" {
return false, errorutil.New(validateResp.Error.Name)
return false, errorutil.New("%s", validateResp.Error.Name)
}
return false, errorutil.New("template validation failed")
}
+23 -23
View File
@@ -19,13 +19,13 @@ require (
github.com/miekg/dns v1.1.59
github.com/olekukonko/tablewriter v0.0.5
github.com/pkg/errors v0.9.1
github.com/projectdiscovery/clistats v0.0.20
github.com/projectdiscovery/fastdialer v0.2.3
github.com/projectdiscovery/hmap v0.0.54
github.com/projectdiscovery/clistats v0.1.0
github.com/projectdiscovery/fastdialer v0.2.7
github.com/projectdiscovery/hmap v0.0.56
github.com/projectdiscovery/interactsh v1.2.0
github.com/projectdiscovery/rawhttp v0.1.60
github.com/projectdiscovery/retryabledns v1.0.69
github.com/projectdiscovery/retryablehttp-go v1.0.71
github.com/projectdiscovery/rawhttp v0.1.61
github.com/projectdiscovery/retryabledns v1.0.73
github.com/projectdiscovery/retryablehttp-go v1.0.76
github.com/projectdiscovery/yamldoc-go v1.0.4
github.com/remeh/sizedwaitgroup v1.0.0
github.com/rs/xid v1.5.0
@@ -36,10 +36,10 @@ require (
github.com/syndtr/goleveldb v1.0.0
github.com/valyala/fasttemplate v1.2.2
github.com/weppos/publicsuffix-go v0.30.2
github.com/xanzy/go-gitlab v0.84.0
github.com/xanzy/go-gitlab v0.107.0
go.uber.org/multierr v1.11.0
golang.org/x/net v0.26.0
golang.org/x/oauth2 v0.18.0
golang.org/x/oauth2 v0.22.0
golang.org/x/text v0.16.0
gopkg.in/yaml.v2 v2.4.0
)
@@ -65,12 +65,14 @@ require (
github.com/docker/go-units v0.5.0
github.com/dop251/goja v0.0.0-20240220182346-e401ed450204
github.com/fatih/structs v1.1.0
github.com/getkin/kin-openapi v0.123.0
github.com/getkin/kin-openapi v0.126.0
github.com/go-git/go-git/v5 v5.11.0
github.com/go-ldap/ldap/v3 v3.4.5
github.com/go-pg/pg v8.0.7+incompatible
github.com/go-sql-driver/mysql v1.7.1
github.com/h2non/filetype v1.1.3
github.com/invopop/yaml v0.3.1
github.com/kitabisa/go-ci v1.0.2
github.com/labstack/echo/v4 v4.10.2
github.com/leslie-qiwa/flat v0.0.0-20230424180412-f9d1cf014baa
github.com/lib/pq v1.10.9
@@ -79,26 +81,27 @@ require (
github.com/microsoft/go-mssqldb v1.6.0
github.com/ory/dockertest/v3 v3.10.0
github.com/praetorian-inc/fingerprintx v1.1.9
github.com/projectdiscovery/dsl v0.1.8
github.com/projectdiscovery/dsl v0.2.1
github.com/projectdiscovery/fasttemplate v0.0.2
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb
github.com/projectdiscovery/goflags v0.1.62
github.com/projectdiscovery/gologger v1.1.18
github.com/projectdiscovery/goflags v0.1.64
github.com/projectdiscovery/gologger v1.1.21
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/gozero v0.0.2
github.com/projectdiscovery/httpx v1.6.7
github.com/projectdiscovery/httpx v1.6.8
github.com/projectdiscovery/mapcidr v1.1.34
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5
github.com/projectdiscovery/ratelimit v0.0.49
github.com/projectdiscovery/ratelimit v0.0.53
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917
github.com/projectdiscovery/sarif v0.0.1
github.com/projectdiscovery/tlsx v1.1.6
github.com/projectdiscovery/tlsx v1.1.7
github.com/projectdiscovery/uncover v1.0.9
github.com/projectdiscovery/useragent v0.0.65
github.com/projectdiscovery/utils v0.2.4
github.com/projectdiscovery/utils v0.2.7
github.com/projectdiscovery/wappalyzergo v0.1.14
github.com/redis/go-redis/v9 v9.1.0
github.com/seh-msft/burpxml v1.0.1
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466
github.com/stretchr/testify v1.9.0
github.com/tarunKoyalwar/goleak v0.0.0-20240429141123-0efa90dbdcf9
github.com/zmap/zgrab2 v0.1.8-0.20230806160807-97ba87c0e706
@@ -152,8 +155,8 @@ require (
github.com/gin-gonic/gin v1.9.1 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.4 // indirect
github.com/go-fed/httpsig v1.1.0 // indirect
github.com/go-openapi/jsonpointer v0.20.2 // indirect
github.com/go-openapi/swag v0.22.9 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
@@ -168,7 +171,6 @@ require (
github.com/hashicorp/go-version v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.6 // indirect
github.com/hbakhtiyor/strsim v0.0.0-20190107154042-4d2bbb273edf // indirect
github.com/invopop/yaml v0.2.0 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
github.com/jcmturner/gofork v1.7.6 // indirect
@@ -262,7 +264,6 @@ require (
github.com/gobwas/pool v0.2.1 // indirect
github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/uuid v1.6.0 // indirect
@@ -306,10 +307,9 @@ require (
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842
golang.org/x/mod v0.17.0 // indirect
golang.org/x/sys v0.21.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/time v0.6.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/protobuf v1.33.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/alecthomas/kingpin.v2 v2.2.6 // indirect
gopkg.in/corvus-ch/zbase32.v1 v1.0.0 // indirect
)
+46 -46
View File
@@ -329,8 +329,8 @@ github.com/gaissmai/bart v0.9.5 h1:vy+r4Px6bjZ+v2QYXAsg63vpz9IfzdW146A8Cn4GPIo=
github.com/gaissmai/bart v0.9.5/go.mod h1:KHeYECXQiBjTzQz/om2tqn3sZF1J7hw9m6z41ftj3fg=
github.com/geoffgarside/ber v1.1.0 h1:qTmFG4jJbwiSzSXoNJeHcOprVzZ8Ulde2Rrrifu5U9w=
github.com/geoffgarside/ber v1.1.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
github.com/getkin/kin-openapi v0.123.0 h1:zIik0mRwFNLyvtXK274Q6ut+dPh6nlxBp0x7mNrPhs8=
github.com/getkin/kin-openapi v0.123.0/go.mod h1:wb1aSZA/iWmorQP9KTAS/phLj/t17B5jT7+fS8ed9NM=
github.com/getkin/kin-openapi v0.126.0 h1:c2cSgLnAsS0xYfKsgt5oBV6MYRM/giU8/RtwUY4wyfY=
github.com/getkin/kin-openapi v0.126.0/go.mod h1:7mONz8IwmSRg6RttPu6v8U/OJ+gr+J99qSFNjPGSQqw=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
@@ -369,10 +369,10 @@ github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG
github.com/go-logfmt/logfmt v0.5.1/go.mod h1:WYhtIu8zTZfxdn5+rREduYbwxfcBr/Vr6KEVveWlfTs=
github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-openapi/jsonpointer v0.20.2 h1:mQc3nmndL8ZBzStEo3JYF8wzmeWffDH4VbXz58sAx6Q=
github.com/go-openapi/jsonpointer v0.20.2/go.mod h1:bHen+N0u1KEO3YlmqOjTT9Adn1RfD91Ar825/PuiRVs=
github.com/go-openapi/swag v0.22.9 h1:XX2DssF+mQKM2DHsbgZK74y/zj4mo9I99+89xUmuZCE=
github.com/go-openapi/swag v0.22.9/go.mod h1:3/OXnFfnMAwBD099SwYRk7GD3xOrr1iL7d/XNLXVVwE=
github.com/go-openapi/jsonpointer v0.21.0 h1:YgdVicSA9vH5RiHs9TZW5oyafXZFc6+2Vc1rr/O9oNQ=
github.com/go-openapi/jsonpointer v0.21.0/go.mod h1:IUyH9l/+uyhIYQ/PXVA41Rexl+kOkAPDdXEYns6fzUY=
github.com/go-openapi/swag v0.23.0 h1:vsEVJDUo2hPJ2tu0/Xc+4noaxyEffXNIs3cOULZ+GrE=
github.com/go-openapi/swag v0.23.0/go.mod h1:esZ8ITTYEsH1V2trKHjAN8Ai7xHb8RV+YSZ577vPjgQ=
github.com/go-pg/pg v8.0.7+incompatible h1:ty/sXL1OZLo+47KK9N8llRcmbA9tZasqbQ/OO4ld53g=
github.com/go-pg/pg v8.0.7+incompatible/go.mod h1:a2oXow+aFOrvwcKs3eIA0lNFmMilrxK2sOkB5NWe0vA=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
@@ -453,8 +453,6 @@ github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.2/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
@@ -572,8 +570,8 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/invopop/jsonschema v0.12.0 h1:6ovsNSuvn9wEQVOyc72aycBMVQFKz7cPdMJn10CvzRI=
github.com/invopop/jsonschema v0.12.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
github.com/invopop/yaml v0.2.0 h1:7zky/qH+O0DwAyoobXUqvVBwgBFRxKoQ/3FjcVpjTMY=
github.com/invopop/yaml v0.2.0/go.mod h1:2XuRLgs/ouIrW3XNzuNj7J3Nvu/Dig5MXvbCEdiBN3Q=
github.com/invopop/yaml v0.3.1 h1:f0+ZpmhfBSS4MhG+4HYseMdJhoeeopbSKbq5Rpeelso=
github.com/invopop/yaml v0.3.1/go.mod h1:PMOp3nn4/12yEZUFfmOuNHJsZToEEOwoWsT+D81KkeA=
github.com/itchyny/gojq v0.12.13 h1:IxyYlHYIlspQHHTE0f3cJF0NKDMfajxViuhBLnHd/QU=
github.com/itchyny/gojq v0.12.13/go.mod h1:JzwzAqenfhrPUuwbmEz3nu3JQmFLlQTQMUcOdnu/Sf4=
github.com/itchyny/timefmt-go v0.1.5 h1:G0INE2la8S6ru/ZI5JecgyzbbJNs5lG1RcBqa7Jm6GE=
@@ -627,6 +625,8 @@ github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kitabisa/go-ci v1.0.2 h1:rqHf8KEbQOxVb998TbqGRo70Z7ol44io7/jLYJUvKp8=
github.com/kitabisa/go-ci v1.0.2/go.mod h1:e3wBSzaJbcifXrr/Gw2ZBLn44MmeqP5WySwXyHlCK/U=
github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.11.4/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.17.8 h1:YcnTYrq7MikUT7k0Yb5eceMmALQPYBW/Xltxn0NAMnU=
@@ -830,30 +830,30 @@ github.com/projectdiscovery/blackrock v0.0.1 h1:lHQqhaaEFjgf5WkuItbpeCZv2DUIE45k
github.com/projectdiscovery/blackrock v0.0.1/go.mod h1:ANUtjDfaVrqB453bzToU+YB4cUbvBRpLvEwoWIwlTss=
github.com/projectdiscovery/cdncheck v1.1.0 h1:qDITidmJsejzpk3rMkauCh6sjI2GH9hW/snk0cQ3kXE=
github.com/projectdiscovery/cdncheck v1.1.0/go.mod h1:sZ8U4MjHSsyaTVjBbYWHT1cwUVvUYwDX1W+WvWRicIc=
github.com/projectdiscovery/clistats v0.0.20 h1:5jO5SLiRJ7f0nDV0ndBNmBeesbROouPooH+DGMgoWq4=
github.com/projectdiscovery/clistats v0.0.20/go.mod h1:GJ2av0KnOvK0AISQnP8hyDclYIji1LVkx2l0pwnzAu4=
github.com/projectdiscovery/dsl v0.1.8 h1:ulz+o097XsYgqP4QXaAQhVSkoeD2ZPWd29wX3CTodUA=
github.com/projectdiscovery/dsl v0.1.8/go.mod h1:AYJS2WQ/q0smr2v4pEJTg4DPe6k56KFKR7UFXvzNz/4=
github.com/projectdiscovery/fastdialer v0.2.3 h1:K03x5XEXGyVWxS2rtSR104E9kHF0aphN7kOCzbh8zv0=
github.com/projectdiscovery/fastdialer v0.2.3/go.mod h1:a0BKvETrO1EAohUYp9gwtbbce0hKD1qGrTegyAUqyRo=
github.com/projectdiscovery/clistats v0.1.0 h1:b+LF1w0xhNd7cneKWMXb+/yUmF1n5szawxP4XGpmbxs=
github.com/projectdiscovery/clistats v0.1.0/go.mod h1:GJ2av0KnOvK0AISQnP8hyDclYIji1LVkx2l0pwnzAu4=
github.com/projectdiscovery/dsl v0.2.1 h1:TK3KD4jsg4YbvY7WJqnz1QyH4AOvAwezeBFOX97Evgk=
github.com/projectdiscovery/dsl v0.2.1/go.mod h1:IRQXsmi5/g1dDZ79//A9t2vrRtxm4frRSd5t8CZVSbI=
github.com/projectdiscovery/fastdialer v0.2.7 h1:x7bEf/kDyVOuG9kHfJLMhWO3h7J+6fMDgLWkGYL8EFg=
github.com/projectdiscovery/fastdialer v0.2.7/go.mod h1:7/BGdaw7sCekuBh6lLZqPuoO5N0Qaizwvex/PjO+V8o=
github.com/projectdiscovery/fasttemplate v0.0.2 h1:h2cISk5xDhlJEinlBQS6RRx0vOlOirB2y3Yu4PJzpiA=
github.com/projectdiscovery/fasttemplate v0.0.2/go.mod h1:XYWWVMxnItd+r0GbjA1GCsUopMw1/XusuQxdyAIHMCw=
github.com/projectdiscovery/freeport v0.0.5 h1:jnd3Oqsl4S8n0KuFkE5Hm8WGDP24ITBvmyw5pFTHS8Q=
github.com/projectdiscovery/freeport v0.0.5/go.mod h1:PY0bxSJ34HVy67LHIeF3uIutiCSDwOqKD8ruBkdiCwE=
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb h1:rutG906Drtbpz4DwU5mhGIeOhRcktDH4cGQitGUMAsg=
github.com/projectdiscovery/go-smb2 v0.0.0-20240129202741-052cc450c6cb/go.mod h1:FLjF1DmZ+POoGEiIQdWuYVwS++C/GwpX8YaCsTSm1RY=
github.com/projectdiscovery/goflags v0.1.62 h1:UmzKJQT+1UyqT1cZDmb3vZ8/IGhQ7LTsWfdqVcAGoJc=
github.com/projectdiscovery/goflags v0.1.62/go.mod h1:d1/D8GaTDoV332ABwceUcY1ffKODaYFlGP0Oriq3wfk=
github.com/projectdiscovery/gologger v1.1.18 h1:fux2S/LXYe18/w4PylAFpEFg+EApNCuyS85x25nDE98=
github.com/projectdiscovery/gologger v1.1.18/go.mod h1:IgjKgVLfVGUovxlC6WJcXK0qvo/tNZ1P0+r6miQqyRk=
github.com/projectdiscovery/goflags v0.1.64 h1:FDfwdt9N97Hi8OuhbkDlKtVttpc/CRMIWQVa08VsHsI=
github.com/projectdiscovery/goflags v0.1.64/go.mod h1:3FyHIVQtnycNOc1LE3O1jj/XR5XuMdF9QfHd0ujhnX4=
github.com/projectdiscovery/gologger v1.1.21 h1:FD9MGE9adY/y2IxDpQgPZIvZAkQIx3bvcuY+d1FLQUo=
github.com/projectdiscovery/gologger v1.1.21/go.mod h1:eyvlcHiSuzJQtwmh+fcfwZVaqVgSGtEP1oL1gJFgP3c=
github.com/projectdiscovery/gostruct v0.0.2 h1:s8gP8ApugGM4go1pA+sVlPDXaWqNP5BBDDSv7VEdG1M=
github.com/projectdiscovery/gostruct v0.0.2/go.mod h1:H86peL4HKwMXcQQtEa6lmC8FuD9XFt6gkNR0B/Mu5PE=
github.com/projectdiscovery/gozero v0.0.2 h1:8fJeaCjxL9tpm33uG/RsCQs6HGM/NE6eA3cjkilRQ+E=
github.com/projectdiscovery/gozero v0.0.2/go.mod h1:d8bZvDWW07LWNYWrwjZ4OO1I0cpkfqaysyDfSs9ibK8=
github.com/projectdiscovery/hmap v0.0.54 h1:b3pdQZwCw4is3xiL2jBx7SJZcYaf/7vtozY7bjUzO/s=
github.com/projectdiscovery/hmap v0.0.54/go.mod h1:j0oakxYOWEfk29wRq5gQgrCv1JnfAfzGaMsRWwEas80=
github.com/projectdiscovery/httpx v1.6.7 h1:luJAtQ+iKpiyKKDpcWw1/fF1MaRX6JZ4R3P+ol5Uikk=
github.com/projectdiscovery/httpx v1.6.7/go.mod h1:rqcuexBrb4v36ZyX0BHcHaovTurOJn2P65Tdt9cYdfE=
github.com/projectdiscovery/hmap v0.0.56 h1:QLIuFhCc7jrBH9WYpxNBDooSoH+/S8cCm7VO5rMt3lU=
github.com/projectdiscovery/hmap v0.0.56/go.mod h1:XyGKMO93nLOzpaTTCDBSKEGx/XWXjSpoyZJxT92Qd/Q=
github.com/projectdiscovery/httpx v1.6.8 h1:k0Y5g3ue/7QbDP0+LykIxp/VhPDLfau3UEUyuxtP7qE=
github.com/projectdiscovery/httpx v1.6.8/go.mod h1:7BIsDxyRwkBjthqFmEajXrA5f3yb4tlVfLmpNdf0ZXA=
github.com/projectdiscovery/interactsh v1.2.0 h1:Al6jHiR+Usl9egYJDLJaWNHOcH8Rugk8gWMasc8Cmw8=
github.com/projectdiscovery/interactsh v1.2.0/go.mod h1:Wxt0fnzxsfrAZQQlpVrf3xMatP4OXZaZbjuDkIQKdYY=
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb h1:MGtI4oE12ruWv11ZlPXXd7hl/uAaQZrFvrIDYDeVMd8=
@@ -866,28 +866,28 @@ github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5 h1:L/e8z8yw
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5/go.mod h1:pGW2ncnTxTxHtP9wzcIJAB+3/NMp6IiuQWd2NK7K+oc=
github.com/projectdiscovery/networkpolicy v0.0.9 h1:IrlDoYZagNNO8y+7iZeHT8k5izE+nek7TdtvEBwCxqk=
github.com/projectdiscovery/networkpolicy v0.0.9/go.mod h1:XFJ2Lnv8BE/ziQCFjBHMsH1w6VmkPiQtk+NlBpdMU7M=
github.com/projectdiscovery/ratelimit v0.0.49 h1:PYatMp8g5OuoFsZOA90e48nLd2vB6a4Tw0FZ8h9zqkQ=
github.com/projectdiscovery/ratelimit v0.0.49/go.mod h1:Xi0LTMHg4HQlmCZFzRBIhRW6N+QW5RxQ8V/Qs+Vta4k=
github.com/projectdiscovery/rawhttp v0.1.60 h1:L57kAATGRuCfOp8Fs0ly3j2wlip7R5pco2RVfwqh/II=
github.com/projectdiscovery/rawhttp v0.1.60/go.mod h1:M97w6lsphTHKeqLSio4Lw9K4KLO2riJMYAOqVMIZ3ck=
github.com/projectdiscovery/ratelimit v0.0.53 h1:1ci2zaVCtJ9daDi9IXjNwpDxl2InYq375role/xqUR4=
github.com/projectdiscovery/ratelimit v0.0.53/go.mod h1:4kNYTL7xKLo1qMYHTREUCBqHyvykFF+XBWutTh9fkwg=
github.com/projectdiscovery/rawhttp v0.1.61 h1:EbskCj6kkDSG31sO5zEUFTqHp9ltccG1DdcI+MCdahQ=
github.com/projectdiscovery/rawhttp v0.1.61/go.mod h1:5XmDAKph9pLVnh87zjL+vXDpfG5W8Gz5N2BAtYTkuVU=
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917 h1:m03X4gBVSorSzvmm0bFa7gDV4QNSOWPL/fgZ4kTXBxk=
github.com/projectdiscovery/rdap v0.9.1-0.20221108103045-9865884d1917/go.mod h1:JxXtZC9e195awe7EynrcnBJmFoad/BNDzW9mzFkK8Sg=
github.com/projectdiscovery/retryabledns v1.0.69 h1:6vhhlfxPQoJI0j1enYSzyagyAZJNfGSg9TNloVsxEBQ=
github.com/projectdiscovery/retryabledns v1.0.69/go.mod h1:5SDuGtXgL4W1z/9V+xnmHFYg03Fi1Ud/F2kqlPlAgfE=
github.com/projectdiscovery/retryablehttp-go v1.0.71 h1:yXPNShCOwoTz7dBSJsBhBh4g4ujX62XS/BrH/fL1VyE=
github.com/projectdiscovery/retryablehttp-go v1.0.71/go.mod h1:wY3T89EwcCKAw6iyMDvwzGPyL3d8TaBU80hnDErEKgM=
github.com/projectdiscovery/retryabledns v1.0.73 h1:CLqtgoI6+YVcUE2jnZPFVvwj59CHDE4HNImLf62sSuw=
github.com/projectdiscovery/retryabledns v1.0.73/go.mod h1:Wo5tQHKtx6XcYYhMypRg8dr0GGnqIb4CzoTSEw2lo7I=
github.com/projectdiscovery/retryablehttp-go v1.0.76 h1:2+4QpWnXUZFBkoeCEBXtUjSeBTdgKtaGwP8LRtKAC3g=
github.com/projectdiscovery/retryablehttp-go v1.0.76/go.mod h1:0qIv7Ld8WWSKWYXMsNAWRYU5gsxnxpBza3SjDgt6nVc=
github.com/projectdiscovery/sarif v0.0.1 h1:C2Tyj0SGOKbCLgHrx83vaE6YkzXEVrMXYRGLkKCr/us=
github.com/projectdiscovery/sarif v0.0.1/go.mod h1:cEYlDu8amcPf6b9dSakcz2nNnJsoz4aR6peERwV+wuQ=
github.com/projectdiscovery/stringsutil v0.0.2 h1:uzmw3IVLJSMW1kEg8eCStG/cGbYYZAja8BH3LqqJXMA=
github.com/projectdiscovery/stringsutil v0.0.2/go.mod h1:EJ3w6bC5fBYjVou6ryzodQq37D5c6qbAYQpGmAy+DC0=
github.com/projectdiscovery/tlsx v1.1.6 h1:iw2zwKbd2+kRQ8J1G4dLmS0CLyemd/tKz1UzcNsC77A=
github.com/projectdiscovery/tlsx v1.1.6/go.mod h1:s7SRRFdrwIZBK/RXXZi4CR/CubqFSvp8h5Bk1srEZIo=
github.com/projectdiscovery/tlsx v1.1.7 h1:eSsl/SmTDL/z2CMeSrbssk4f/9oOotMP1SgXl3yynSM=
github.com/projectdiscovery/tlsx v1.1.7/go.mod h1:g66QQ4/y4tLVjoGbzWIv+Q6xwFzxfJbEDx86Y1dYHDM=
github.com/projectdiscovery/uncover v1.0.9 h1:s5RbkD/V4r8QcPkys4gTTqMuRSgXq0JprejqLSopN9Y=
github.com/projectdiscovery/uncover v1.0.9/go.mod h1:2PUF3SpB5QNIJ8epaB2xbRzkPaxEAWRDm3Ir2ijt81U=
github.com/projectdiscovery/useragent v0.0.65 h1:x78ZwWdqpzokOHxLITUXvq+ljkTKc19z3ILGtoV1N70=
github.com/projectdiscovery/useragent v0.0.65/go.mod h1:deOP8YLJU6SCzM8k+K8PjkcOF4Ux0spqyO4ODZGIT4A=
github.com/projectdiscovery/utils v0.2.4 h1:CHnlt2la4jr8TeL7ZK7UhQItHY7DDXqIuLnnxyAJLDY=
github.com/projectdiscovery/utils v0.2.4/go.mod h1:2Vx7geSrBfCPqknZywqbChQm8SE30mcyrlB5YsxEnhA=
github.com/projectdiscovery/utils v0.2.7 h1:XWdz7SscL++jqsnQ9ecHzSZE0RK33tyPcnqcXw+vmKs=
github.com/projectdiscovery/utils v0.2.7/go.mod h1:N0N7tbdNFPegd9NpJ3onCPClaBrERcOIB88yww6UCF8=
github.com/projectdiscovery/wappalyzergo v0.1.14 h1:nt1IM4RUmqeymsXk4h6BsZbKDoS2hjFvPkT2GaI1rz4=
github.com/projectdiscovery/wappalyzergo v0.1.14/go.mod h1:/hzgxkBFTMe2wDbA93nFfoMjULw7/vIZ9QPSAnCgUa8=
github.com/projectdiscovery/yamldoc-go v1.0.4 h1:eZoESapnMw6WAHiVgRwNqvbJEfNHEH148uthhFbG5jE=
@@ -964,6 +964,8 @@ github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFt
github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466 h1:17JxqqJY66GmZVHkmAsGEkcIu0oCe3AM420QDgGwZx0=
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466/go.mod h1:9dIRpgIY7hVhoqfe0/FcYp0bpInZaT7dc3BYOprrIUE=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.3.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
@@ -1068,8 +1070,8 @@ github.com/weppos/publicsuffix-go v0.30.2/go.mod h1:/hGscit36Yt+wammfBBwdMdxBT8b
github.com/weppos/publicsuffix-go/publicsuffix/generator v0.0.0-20220927085643-dc0d00c92642/go.mod h1:GHfoeIdZLdZmLjMlzBftbTDntahTttUMWjxZwQJhULE=
github.com/wk8/go-ordered-map/v2 v2.1.8 h1:5h/BUHu93oj4gIdvHHHGsScSTMijfx5PeYkE/fJgbpc=
github.com/wk8/go-ordered-map/v2 v2.1.8/go.mod h1:5nJHM5DyteebpVlHnWMV0rPz6Zp7+xBAnxjb1X5vnTw=
github.com/xanzy/go-gitlab v0.84.0 h1:PdpCaskQSgcVDsx21c6ikf8Rfyo7SNtFAJwP9PrbCFE=
github.com/xanzy/go-gitlab v0.84.0/go.mod h1:5ryv+MnpZStBH8I/77HuQBsMbBGANtVpLWC15qOjWAw=
github.com/xanzy/go-gitlab v0.107.0 h1:P2CT9Uy9yN9lJo3FLxpMZ4xj6uWcpnigXsjvqJ6nd2Y=
github.com/xanzy/go-gitlab v0.107.0/go.mod h1:wKNKh3GkYDMOsGmnfuX+ITCmDuSDWFO0G+C4AygL9RY=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
@@ -1291,8 +1293,8 @@ golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ
golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc=
golang.org/x/oauth2 v0.5.0/go.mod h1:9/XBHVqLaWO3/BRHs5jbpYCnOZVjj5V0ndyaAM7KB4I=
golang.org/x/oauth2 v0.6.0/go.mod h1:ycmewcwgD4Rpr3eZJLSB4Kyyljb3qDh40vJ8STE5HKw=
golang.org/x/oauth2 v0.18.0 h1:09qnuIAgzdx1XplqJvW6CQqMCtGZykZWcXzPMPUusvI=
golang.org/x/oauth2 v0.18.0/go.mod h1:Wf7knwG0MPoWIMMBgFlEaSUDaKskp0dCfrlJRJXbBi8=
golang.org/x/oauth2 v0.22.0 h1:BzDx2FehcG7jJwgWLELCdmLuxk2i+x9UDpSiss2u0ZA=
golang.org/x/oauth2 v0.22.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -1421,8 +1423,8 @@ golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxb
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/time v0.6.0 h1:eTDhh4ZXt5Qf0augr54TN6suAUudPcawVZeIAPU7D4U=
golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -1505,8 +1507,6 @@ google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
@@ -1569,8 +1569,9 @@ google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQ
google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/alecthomas/kingpin.v2 v2.2.6 h1:jMFz6MfLP0/4fUyZle81rXUoxOBFi19VUFKVDOQfozc=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -1604,7 +1605,6 @@ gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.3.0 h1:MfDY1b1/0xN1CyMlQDac0ziEy9zJQd9CXBRRDHw2jJo=
+9 -1
View File
@@ -5,9 +5,17 @@ import (
urlutil "github.com/projectdiscovery/utils/url"
)
func getScanDashBoardURL(id string) string {
func getScanDashBoardURL(id string, teamID string) string {
ux, _ := urlutil.Parse(pdcpauth.DashBoardURL)
ux.Path = "/scans/" + id
if ux.Params == nil {
ux.Params = urlutil.NewOrderedParams()
}
if teamID != "" {
ux.Params.Add("team_id", teamID)
} else {
ux.Params.Add("team_id", NoneTeamID)
}
ux.Update()
return ux.String()
}
+18 -7
View File
@@ -32,13 +32,14 @@ const (
MaxChunkSize = 4 * unitutils.Mega // 4 MB
xidRe = `^[a-z0-9]{20}$`
teamIDHeader = "X-Team-Id"
NoneTeamID = "none"
)
var (
xidRegex = regexp.MustCompile(xidRe)
_ output.Writer = &UploadWriter{}
// teamID if given
teamID = env.GetEnvOrDefault("PDCP_TEAM_ID", "")
TeamIDEnv = env.GetEnvOrDefault("PDCP_TEAM_ID", NoneTeamID)
)
// UploadWriter is a writer that uploads its output to pdcp
@@ -53,6 +54,7 @@ type UploadWriter struct {
scanID string
scanName string
counter atomic.Int32
TeamID string
}
// NewUploadWriter creates a new upload writer
@@ -61,8 +63,9 @@ func NewUploadWriter(ctx context.Context, creds *pdcpauth.PDCPCredentials) (*Upl
return nil, fmt.Errorf("no credentials provided")
}
u := &UploadWriter{
creds: creds,
done: make(chan struct{}, 1),
creds: creds,
done: make(chan struct{}, 1),
TeamID: NoneTeamID,
}
var err error
reader, writer := io.Pipe()
@@ -110,6 +113,14 @@ func (u *UploadWriter) SetScanName(name string) {
u.scanName = name
}
func (u *UploadWriter) SetTeamID(id string) {
if id == "" {
u.TeamID = NoneTeamID
} else {
u.TeamID = id
}
}
func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
reader := bufio.NewReader(r)
ch := make(chan string, 4)
@@ -136,7 +147,7 @@ func (u *UploadWriter) autoCommit(ctx context.Context, r *io.PipeReader) {
if u.scanID == "" {
gologger.Verbose().Msgf("Scan results upload to cloud skipped, no results found to upload")
} else {
gologger.Info().Msgf("%v Scan results uploaded to cloud, you can view scan results at %v", u.counter.Load(), getScanDashBoardURL(u.scanID))
gologger.Info().Msgf("%v Scan results uploaded to cloud, you can view scan results at %v", u.counter.Load(), getScanDashBoardURL(u.scanID, u.TeamID))
}
}()
// temporary buffer to store the results
@@ -189,7 +200,7 @@ func (u *UploadWriter) uploadChunk(buff *bytes.Buffer) error {
// if successful, reset the buffer
buff.Reset()
// log in verbose mode
gologger.Warning().Msgf("Uploaded results chunk, you can view scan results at %v", getScanDashBoardURL(u.scanID))
gologger.Warning().Msgf("Uploaded results chunk, you can view scan results at %v", getScanDashBoardURL(u.scanID, u.TeamID))
return nil
}
@@ -248,8 +259,8 @@ func (u *UploadWriter) getRequest(bin []byte) (*retryablehttp.Request, error) {
req.URL.Update()
req.Header.Set(pdcpauth.ApiKeyHeaderName, u.creds.APIKey)
if teamID != "" {
req.Header.Set(teamIDHeader, teamID)
if u.TeamID != NoneTeamID && u.TeamID != "" {
req.Header.Set(teamIDHeader, u.TeamID)
}
req.Header.Set("Content-Type", "application/octet-stream")
req.Header.Set("Accept", "application/json")
+3
View File
@@ -426,6 +426,9 @@ func (r *Runner) setupPDCPUpload(writer output.Writer) output.Writer {
if r.options.ScanName != "" {
uploadWriter.SetScanName(r.options.ScanName)
}
if r.options.TeamID != "" {
uploadWriter.SetTeamID(r.options.TeamID)
}
return output.NewMultiWriter(writer, uploadWriter)
}
+1 -1
View File
@@ -82,5 +82,5 @@ func TestMain(m *testing.M) {
// no need to run this test on github actions
return
}
m.Run()
os.Exit(m.Run())
}
+2 -1
View File
@@ -178,7 +178,8 @@ func (e *NucleiEngine) SignTemplate(tmplSigner *signer.TemplateSigner, data []by
if err != nil {
return data, err
}
buff := bytes.NewBuffer(signer.RemoveSignatureFromData(data))
_, content := signer.ExtractSignatureAndContent(data)
buff := bytes.NewBuffer(content)
buff.WriteString("\n" + signatureData)
return buff.Bytes(), err
}
+6
View File
@@ -256,6 +256,7 @@
"deleteheader",
"setbody",
"waitevent",
"waitdialog",
"keyboard",
"debug",
"sleep",
@@ -787,6 +788,11 @@
"title": "signature is the http request signature method",
"description": "Signature is the HTTP Request signature Method"
},
"skip-secret-file": {
"type": "boolean",
"title": "bypass secret file",
"description": "Skips the authentication or authorization configured in the secret file"
},
"cookie-reuse": {
"type": "boolean",
"title": "optional cookie reuse enable",
+47 -31
View File
@@ -16,8 +16,8 @@ import (
type FileAuthProvider struct {
Path string
store *authx.Authx
compiled map[*regexp.Regexp]authx.AuthStrategy
domains map[string]authx.AuthStrategy
compiled map[*regexp.Regexp][]authx.AuthStrategy
domains map[string][]authx.AuthStrategy
}
// NewFileAuthProvider creates a new file based auth provider
@@ -56,25 +56,31 @@ func (f *FileAuthProvider) init() {
if len(secret.DomainsRegex) > 0 {
for _, domain := range secret.DomainsRegex {
if f.compiled == nil {
f.compiled = make(map[*regexp.Regexp]authx.AuthStrategy)
f.compiled = make(map[*regexp.Regexp][]authx.AuthStrategy)
}
compiled, err := regexp.Compile(domain)
if err != nil {
continue
}
f.compiled[compiled] = secret.GetStrategy()
if ss, ok := f.compiled[compiled]; ok {
f.compiled[compiled] = append(ss, secret.GetStrategy())
} else {
f.compiled[compiled] = []authx.AuthStrategy{secret.GetStrategy()}
}
}
}
for _, domain := range secret.Domains {
if f.domains == nil {
f.domains = make(map[string]authx.AuthStrategy)
f.domains = make(map[string][]authx.AuthStrategy)
}
f.domains[strings.TrimSpace(domain)] = secret.GetStrategy()
if strings.HasSuffix(domain, ":80") {
f.domains[strings.TrimSuffix(domain, ":80")] = secret.GetStrategy()
}
if strings.HasSuffix(domain, ":443") {
f.domains[strings.TrimSuffix(domain, ":443")] = secret.GetStrategy()
domain = strings.TrimSpace(domain)
domain = strings.TrimSuffix(domain, ":80")
domain = strings.TrimSuffix(domain, ":443")
if ss, ok := f.domains[domain]; ok {
f.domains[domain] = append(ss, secret.GetStrategy())
} else {
f.domains[domain] = []authx.AuthStrategy{secret.GetStrategy()}
}
}
}
@@ -82,32 +88,38 @@ func (f *FileAuthProvider) init() {
if len(dynamic.DomainsRegex) > 0 {
for _, domain := range dynamic.DomainsRegex {
if f.compiled == nil {
f.compiled = make(map[*regexp.Regexp]authx.AuthStrategy)
f.compiled = make(map[*regexp.Regexp][]authx.AuthStrategy)
}
compiled, err := regexp.Compile(domain)
if err != nil {
continue
}
f.compiled[compiled] = &authx.DynamicAuthStrategy{Dynamic: dynamic}
if ss, ok := f.compiled[compiled]; !ok {
f.compiled[compiled] = []authx.AuthStrategy{&authx.DynamicAuthStrategy{Dynamic: dynamic}}
} else {
f.compiled[compiled] = append(ss, &authx.DynamicAuthStrategy{Dynamic: dynamic})
}
}
}
for _, domain := range dynamic.Domains {
if f.domains == nil {
f.domains = make(map[string]authx.AuthStrategy)
f.domains = make(map[string][]authx.AuthStrategy)
}
f.domains[strings.TrimSpace(domain)] = &authx.DynamicAuthStrategy{Dynamic: dynamic}
if strings.HasSuffix(domain, ":80") {
f.domains[strings.TrimSuffix(domain, ":80")] = &authx.DynamicAuthStrategy{Dynamic: dynamic}
}
if strings.HasSuffix(domain, ":443") {
f.domains[strings.TrimSuffix(domain, ":443")] = &authx.DynamicAuthStrategy{Dynamic: dynamic}
domain = strings.TrimSpace(domain)
domain = strings.TrimSuffix(domain, ":80")
domain = strings.TrimSuffix(domain, ":443")
if ss, ok := f.domains[domain]; !ok {
f.domains[domain] = []authx.AuthStrategy{&authx.DynamicAuthStrategy{Dynamic: dynamic}}
} else {
f.domains[domain] = append(ss, &authx.DynamicAuthStrategy{Dynamic: dynamic})
}
}
}
}
// LookupAddr looks up a given domain/address and returns appropriate auth strategy
func (f *FileAuthProvider) LookupAddr(addr string) authx.AuthStrategy {
func (f *FileAuthProvider) LookupAddr(addr string) []authx.AuthStrategy {
if strings.Contains(addr, ":") {
// default normalization for host:port
host, port, err := net.SplitHostPort(addr)
@@ -129,12 +141,12 @@ func (f *FileAuthProvider) LookupAddr(addr string) authx.AuthStrategy {
}
// LookupURL looks up a given URL and returns appropriate auth strategy
func (f *FileAuthProvider) LookupURL(u *url.URL) authx.AuthStrategy {
func (f *FileAuthProvider) LookupURL(u *url.URL) []authx.AuthStrategy {
return f.LookupAddr(u.Host)
}
// LookupURLX looks up a given URL and returns appropriate auth strategy
func (f *FileAuthProvider) LookupURLX(u *urlutil.URL) authx.AuthStrategy {
func (f *FileAuthProvider) LookupURLX(u *urlutil.URL) []authx.AuthStrategy {
return f.LookupAddr(u.Host)
}
@@ -151,17 +163,21 @@ func (f *FileAuthProvider) GetTemplatePaths() []string {
// PreFetchSecrets pre-fetches the secrets from the auth provider
func (f *FileAuthProvider) PreFetchSecrets() error {
for _, s := range f.domains {
if val, ok := s.(*authx.DynamicAuthStrategy); ok {
if err := val.Dynamic.Fetch(false); err != nil {
return err
for _, ss := range f.domains {
for _, s := range ss {
if val, ok := s.(*authx.DynamicAuthStrategy); ok {
if err := val.Dynamic.Fetch(false); err != nil {
return err
}
}
}
}
for _, s := range f.compiled {
if val, ok := s.(*authx.DynamicAuthStrategy); ok {
if err := val.Dynamic.Fetch(false); err != nil {
return err
for _, ss := range f.compiled {
for _, s := range ss {
if val, ok := s.(*authx.DynamicAuthStrategy); ok {
if err := val.Dynamic.Fetch(false); err != nil {
return err
}
}
}
}
+3 -3
View File
@@ -22,13 +22,13 @@ var (
type AuthProvider interface {
// LookupAddr looks up a given domain/address and returns appropriate auth strategy
// for it (accepted inputs are scanme.sh or scanme.sh:443)
LookupAddr(string) authx.AuthStrategy
LookupAddr(string) []authx.AuthStrategy
// LookupURL looks up a given URL and returns appropriate auth strategy
// it accepts a valid url struct and returns the auth strategy
LookupURL(*url.URL) authx.AuthStrategy
LookupURL(*url.URL) []authx.AuthStrategy
// LookupURLX looks up a given URL and returns appropriate auth strategy
// it accepts pd url struct (i.e urlutil.URL) and returns the auth strategy
LookupURLX(*urlutil.URL) authx.AuthStrategy
LookupURLX(*urlutil.URL) []authx.AuthStrategy
// GetTemplatePaths returns the template path for the auth provider
// that will be used for dynamic secret fetching
GetTemplatePaths() []string
+3 -3
View File
@@ -19,7 +19,7 @@ func NewMultiAuthProvider(providers ...AuthProvider) AuthProvider {
return &MultiAuthProvider{Providers: providers}
}
func (m *MultiAuthProvider) LookupAddr(host string) authx.AuthStrategy {
func (m *MultiAuthProvider) LookupAddr(host string) []authx.AuthStrategy {
for _, provider := range m.Providers {
strategy := provider.LookupAddr(host)
if strategy != nil {
@@ -29,7 +29,7 @@ func (m *MultiAuthProvider) LookupAddr(host string) authx.AuthStrategy {
return nil
}
func (m *MultiAuthProvider) LookupURL(u *url.URL) authx.AuthStrategy {
func (m *MultiAuthProvider) LookupURL(u *url.URL) []authx.AuthStrategy {
for _, provider := range m.Providers {
strategy := provider.LookupURL(u)
if strategy != nil {
@@ -39,7 +39,7 @@ func (m *MultiAuthProvider) LookupURL(u *url.URL) authx.AuthStrategy {
return nil
}
func (m *MultiAuthProvider) LookupURLX(u *urlutil.URL) authx.AuthStrategy {
func (m *MultiAuthProvider) LookupURLX(u *urlutil.URL) []authx.AuthStrategy {
for _, provider := range m.Providers {
strategy := provider.LookupURLX(u)
if strategy != nil {
+1 -1
View File
@@ -31,7 +31,7 @@ const (
CLIConfigFileName = "config.yaml"
ReportingConfigFilename = "reporting-config.yaml"
// Version is the current version of nuclei
Version = `v3.3.0`
Version = `v3.3.2`
// Directory Names of custom templates
CustomS3TemplatesDirName = "s3"
CustomGitHubTemplatesDirName = "github"
+3 -3
View File
@@ -83,8 +83,8 @@ func (bk *customTemplateGitLabRepo) Download(_ context.Context) {
// Get the directory listing for the files in the project
tree, _, err := bk.gitLabClient.Repositories.ListTree(projectID, &gitlab.ListTreeOptions{
Ref: gitlab.String(project.DefaultBranch),
Recursive: gitlab.Bool(true),
Ref: gitlab.Ptr(project.DefaultBranch),
Recursive: gitlab.Ptr(true),
})
if err != nil {
gologger.Error().Msgf("error retrieving files from GitLab project: %s (%d) %s", project.Name, projectID, err)
@@ -95,7 +95,7 @@ func (bk *customTemplateGitLabRepo) Download(_ context.Context) {
// If the object is not a file or file extension is not .yaml, skip it
if file.Type == "blob" && filepath.Ext(file.Path) == ".yaml" {
gf := &gitlab.GetFileOptions{
Ref: gitlab.String(project.DefaultBranch),
Ref: gitlab.Ptr(project.DefaultBranch),
}
f, _, err := bk.gitLabClient.RepositoryFiles.GetFile(projectID, file.Path, gf)
if err != nil {
+18 -3
View File
@@ -7,6 +7,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/fuzz/dataformat"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/projectdiscovery/utils/maps"
urlutil "github.com/projectdiscovery/utils/url"
)
@@ -81,7 +82,7 @@ func (q *Path) Delete(key string) error {
// Rebuild returns a new request with the
// component rebuilt
func (q *Path) Rebuild() (*retryablehttp.Request, error) {
originalValues := make(map[string]interface{})
originalValues := mapsutil.Map[string, any]{}
splitted := strings.Split(q.req.URL.Path, "/")
for i := range splitted {
pathTillNow := strings.Join(splitted[:i+1], "/")
@@ -95,8 +96,22 @@ func (q *Path) Rebuild() (*retryablehttp.Request, error) {
lengthSplitted := len(q.value.parsed.Map)
for i := lengthSplitted; i > 0; i-- {
key := strconv.Itoa(i)
original := originalValues[key].(string)
new := q.value.parsed.Map[key].(string)
original, ok := originalValues.GetOrDefault(key, "").(string)
if !ok {
continue
}
new, ok := q.value.parsed.Map.GetOrDefault(key, "").(string)
if !ok {
continue
}
if new == original {
// no need to replace
continue
}
originalPath = strings.Replace(originalPath, original, new, 1)
}
+11 -1
View File
@@ -73,12 +73,22 @@ func (v *Value) SetParsed(data dataformat.KV, dataFormat string) {
// SetParsedValue sets the parsed value for a key
// in the parsed map
func (v *Value) SetParsedValue(key string, value string) bool {
func (v *Value) SetParsedValue(key, value string) bool {
if key == "" {
return false
}
origValue := v.parsed.Get(key)
if origValue == nil {
v.parsed.Set(key, value)
return true
}
// TODO(dwisiswant0): I'm sure that this can be simplified because
// `dataformat.KV.*` is a type of `mapsutil.*` where the value is `any`. So,
// it looks like we won't type conversion here or even have its own methods
// inside `dataformat.KV`.
// If the value is a list, append to it
// otherwise replace it
switch v := origValue.(type) {
+1 -1
View File
@@ -13,7 +13,7 @@ import (
// if it's not important/significant (ex: json,xml) we use map
// this also allows us to iteratively implement ordered map
type KV struct {
Map map[string]interface{}
Map mapsutil.Map[string, any]
OrderedMap *mapsutil.OrderedMap[string, any]
}
+12 -1
View File
@@ -16,6 +16,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/retryablehttp-go"
errorutil "github.com/projectdiscovery/utils/errors"
mapsutil "github.com/projectdiscovery/utils/maps"
sliceutil "github.com/projectdiscovery/utils/slice"
urlutil "github.com/projectdiscovery/utils/url"
)
@@ -165,6 +166,11 @@ mainLoop:
func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, interactshUrls []string) (map[string]interface{}, []string) {
// Check if Interactsh options are configured
if rule.options.Interactsh != nil {
interactshUrlsMap := make(map[string]struct{})
for _, url := range interactshUrls {
interactshUrlsMap[url] = struct{}{}
}
interactshUrls = mapsutil.GetKeys(interactshUrlsMap)
// Iterate through the data to replace and evaluate variables with Interactsh URLs
for k, v := range data {
value := fmt.Sprint(v)
@@ -175,7 +181,12 @@ func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, intera
}
// Append new OAST URLs if any
if len(oastUrls) > 0 {
interactshUrls = append(interactshUrls, oastUrls...)
for _, url := range oastUrls {
if _, ok := interactshUrlsMap[url]; !ok {
interactshUrlsMap[url] = struct{}{}
interactshUrls = append(interactshUrls, url)
}
}
}
// Evaluate the replaced data
evaluatedData, err := expressions.Evaluate(got, data)
+7 -7
View File
@@ -18,7 +18,7 @@ func getSchemaExample(schema *openapi3.Schema) (interface{}, bool) {
return schema.Default, true
}
if schema.Enum != nil && len(schema.Enum) > 0 {
if len(schema.Enum) > 0 {
return schema.Enum[0], true
}
return nil, false
@@ -175,9 +175,9 @@ func openAPIExample(schema *openapi3.Schema, cache map[*openapi3.Schema]*cachedS
}
switch {
case schema.Type == "boolean":
case schema.Type.Is("boolean"):
return true, nil
case schema.Type == "number", schema.Type == "integer":
case schema.Type.Is("number"), schema.Type.Is("integer"):
value := 0.0
if schema.Min != nil && *schema.Min > value {
@@ -208,11 +208,11 @@ func openAPIExample(schema *openapi3.Schema, cache map[*openapi3.Schema]*cachedS
value += float64(int(*schema.MultipleOf) - (int(value) % int(*schema.MultipleOf)))
}
if schema.Type == "integer" {
if schema.Type.Is("integer") {
return int(value), nil
}
return value, nil
case schema.Type == "string":
case schema.Type.Is("string"):
if ex := stringFormatExample(schema.Format); ex != "" {
return ex, nil
}
@@ -226,7 +226,7 @@ func openAPIExample(schema *openapi3.Schema, cache map[*openapi3.Schema]*cachedS
example = example[:*schema.MaxLength]
}
return example, nil
case schema.Type == "array", schema.Items != nil:
case schema.Type.Is("array"), schema.Items != nil:
example := []interface{}{}
if schema.Items != nil && schema.Items.Value != nil {
@@ -242,7 +242,7 @@ func openAPIExample(schema *openapi3.Schema, cache map[*openapi3.Schema]*cachedS
}
}
return example, nil
case schema.Type == "object", len(schema.Properties) > 0:
case schema.Type.Is("object"), len(schema.Properties) > 0:
example := map[string]interface{}{}
for k, v := range schema.Properties {
+13 -3
View File
@@ -75,16 +75,26 @@ func GenerateRequestsFromSchema(schema *openapi3.T, opts formats.InputFormatOpti
for _, serverURL := range schema.Servers {
pathURL := serverURL.URL
// Split the server URL into baseURL and serverPath
u, err := url.Parse(pathURL)
if err != nil {
return errors.Wrap(err, "could not parse server url")
}
baseURL := fmt.Sprintf("%s://%s", u.Scheme, u.Host)
serverPath := u.Path
for path, v := range schema.Paths.Map() {
// a path item can have parameters
ops := v.Operations()
requestPath := path
if serverPath != "" {
requestPath = serverPath + path
}
for method, ov := range ops {
if err := generateRequestsFromOp(&generateReqOptions{
requiredOnly: opts.RequiredOnly,
method: method,
pathURL: pathURL,
pathURL: baseURL,
requestPath: requestPath,
op: ov,
schema: schema,
@@ -193,7 +203,7 @@ func generateRequestsFromOp(opts *generateReqOptions) error {
paramValue = value.Schema.Value.Default
} else if value.Schema.Value.Example != nil {
paramValue = value.Schema.Value.Example
} else if value.Schema.Value.Enum != nil && len(value.Schema.Value.Enum) > 0 {
} else if len(value.Schema.Value.Enum) > 0 {
paramValue = value.Schema.Value.Enum[0]
} else {
if !opts.opts.SkipFormatValidation {
@@ -406,7 +416,7 @@ schemaLabel:
return globalParams, nil
}
// generateExampleFromSchema generates an example from a schema object
// GenerateParameterFromSecurityScheme generates an example from a schema object
func GenerateParameterFromSecurityScheme(scheme *openapi3.SecuritySchemeRef) (*openapi3.Parameter, error) {
if !generic.EqualsAny(scheme.Value.Type, "http", "apiKey") {
return nil, errorutil.NewWithTag("openapi", "unsupported security scheme type (%s) found in openapi file", scheme.Value.Type)
+7 -2
View File
@@ -2,6 +2,7 @@ package swagger
import (
"encoding/json"
"io"
"os"
"path"
@@ -10,7 +11,7 @@ import (
"github.com/pkg/errors"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/openapi"
"gopkg.in/yaml.v2"
"github.com/invopop/yaml"
"github.com/getkin/kin-openapi/openapi2conv"
)
@@ -49,7 +50,11 @@ func (j *SwaggerFormat) Parse(input string, resultsCb formats.ParseReqRespCallba
ext := path.Ext(input)
if ext == ".yaml" || ext == ".yml" {
err = yaml.NewDecoder(file).Decode(schemav2)
data, err_data := io.ReadAll(file)
if err_data != nil {
return errors.Wrap(err, "could not read data file")
}
err = yaml.Unmarshal(data, schemav2)
} else {
err = json.NewDecoder(file).Decode(schemav2)
}
+2 -2
View File
@@ -27,8 +27,8 @@ func TestSwaggerAPIParser(t *testing.T) {
}
expectedURLs := []string{
"https://localhost/users",
"https://localhost/users/1?test=asc",
"https://localhost/v1/users",
"https://localhost/v1/users/1?test=asc",
}
require.ElementsMatch(t, gotMethodsToURLs, expectedURLs, "could not get swagger urls")
}
+1 -1
View File
@@ -121,7 +121,7 @@ func NewInputProvider(opts InputOptions) (InputProvider, error) {
}
}
// SupportedFormats returns all supported input formats of nuclei
// SupportedInputFormats returns all supported input formats of nuclei
func SupportedInputFormats() string {
return "list, " + http.SupportedFormats()
}
+7
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"github.com/dop251/goja"
"github.com/kitabisa/go-ci"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -120,11 +121,17 @@ func (c *Compiler) ExecuteWithOptions(program *goja.Program, args *ExecuteArgs,
defer cancel()
// execute the script
results, err := contextutil.ExecFuncWithTwoReturns(ctx, func() (val goja.Value, err error) {
// TODO(dwisiswant0): remove this once we get the RCA.
defer func() {
if ci.IsCI() {
return
}
if r := recover(); r != nil {
err = fmt.Errorf("panic: %v", r)
}
}()
return ExecuteProgram(program, args, opts)
})
if err != nil {
+8
View File
@@ -11,6 +11,7 @@ import (
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/console"
"github.com/dop251/goja_nodejs/require"
"github.com/kitabisa/go-ci"
"github.com/projectdiscovery/gologger"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libbytes"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libfs"
@@ -84,11 +85,18 @@ func executeWithRuntime(runtime *goja.Runtime, p *goja.Program, args *ExecuteArg
opts.Cleanup(runtime)
}
}()
// TODO(dwisiswant0): remove this once we get the RCA.
defer func() {
if ci.IsCI() {
return
}
if r := recover(); r != nil {
err = fmt.Errorf("panic: %s", r)
}
}()
// set template ctx
_ = runtime.Set("template", args.TemplateCtx)
// set args
+1 -1
View File
@@ -249,7 +249,7 @@ func identifyGenDecl(pkg *ast.Package, decl *ast.GenDecl, data *TemplateData) {
if !spec.Names[0].IsExported() {
continue
}
if spec.Values == nil || len(spec.Values) == 0 {
if len(spec.Values) == 0 {
continue
}
data.PackageVars[spec.Names[0].Name] = spec.Names[0].Name
+1 -1
View File
@@ -489,7 +489,7 @@ func (p *EntityParser) extractVarsNConstants() {
if !spec.Names[0].IsExported() {
continue
}
if spec.Values == nil || len(spec.Values) == 0 {
if len(spec.Values) == 0 {
continue
}
// get comments or description
+7 -3
View File
@@ -60,12 +60,16 @@ func (j *NucleiJS) HandleError(err error, msg ...string) {
if len(msg) == 0 {
j.ThrowError(err)
}
j.Throw(fmt.Sprintf("%s: %s", strings.Join(msg, ":"), err.Error()))
j.Throw("%s: %s", strings.Join(msg, ":"), err.Error())
}
// Throw throws an error in goja runtime
func (j *NucleiJS) Throw(format string, args ...interface{}) {
panic(j.runtime().ToValue(fmt.Sprintf(format, args...)))
if len(args) > 0 {
panic(j.runtime().ToValue(fmt.Sprintf(format, args...)))
}
panic(j.runtime().ToValue(format))
}
// GetArg returns argument at index from goja runtime if not found throws error
@@ -95,7 +99,7 @@ func (j *NucleiJS) GetArgSafe(args []goja.Value, index int, defaultValue any) an
// Require throws an error if expression is false
func (j *NucleiJS) Require(expr bool, msg string) {
if !expr {
j.Throw(msg)
j.Throw("%s", msg)
}
}
+19 -2
View File
@@ -71,6 +71,7 @@ type StandardWriter struct {
omitTemplate bool
DisableStdout bool
AddNewLinesOutputFile bool // by default this is only done for stdout
KeysToRedact []string
}
var decolorizerRegex = regexp.MustCompile(`\x1B\[[0-9;]*[a-zA-Z]`)
@@ -253,6 +254,7 @@ func NewStandardWriter(options *types.Options) (*StandardWriter, error) {
storeResponse: options.StoreResponse,
storeResponseDir: options.StoreResponseDir,
omitTemplate: options.OmitTemplate,
KeysToRedact: options.Redact,
}
return writer, nil
}
@@ -264,6 +266,13 @@ func (w *StandardWriter) Write(event *ResultEvent) error {
event.Template, event.TemplateURL = utils.TemplatePathURL(types.ToString(event.TemplatePath), types.ToString(event.TemplateID))
}
if len(w.KeysToRedact) > 0 {
event.Request = redactKeys(event.Request, w.KeysToRedact)
event.Response = redactKeys(event.Response, w.KeysToRedact)
event.CURLCommand = redactKeys(event.CURLCommand, w.KeysToRedact)
event.Matched = redactKeys(event.Matched, w.KeysToRedact)
}
event.Timestamp = time.Now()
var data []byte
@@ -302,6 +311,14 @@ func (w *StandardWriter) Write(event *ResultEvent) error {
return nil
}
func redactKeys(data string, keysToRedact []string) string {
for _, key := range keysToRedact {
keyPattern := regexp.MustCompile(fmt.Sprintf(`(?i)(%s\s*[:=]\s*["']?)[^"'\r\n&]+(["'\r\n]?)`, regexp.QuoteMeta(key)))
data = keyPattern.ReplaceAllString(data, `$1***$2`)
}
return data
}
// JSONLogRequest is a trace/error log request written to file
type JSONLogRequest struct {
Template string `json:"template"`
@@ -508,12 +525,12 @@ func tryParseCause(err error) error {
if strings.HasPrefix(msg, "ReadStatusLine:") {
// last index is actual error (from rawhttp)
parts := strings.Split(msg, ":")
return errkit.New(strings.TrimSpace(parts[len(parts)-1]))
return errkit.New("%s", strings.TrimSpace(parts[len(parts)-1]))
}
if strings.Contains(msg, "read ") {
// same here
parts := strings.Split(msg, ":")
return errkit.New(strings.TrimSpace(parts[len(parts)-1]))
return errkit.New("%s", strings.TrimSpace(parts[len(parts)-1]))
}
return err
}
+1 -1
View File
@@ -17,7 +17,7 @@ func hash(v interface{}) (string, error) {
sh := sha256.New()
if _, err = io.WriteString(sh, string(data)); err != nil {
if _, err = sh.Write(data); err != nil {
return "", err
}
return hex.EncodeToString(sh.Sum(nil)), nil
@@ -44,7 +44,8 @@ func toHighLightedHexDump(hexDump, snippetToHighlight string) (HighlightableHexD
hexDumpRowValues := hexDumpParsePattern.FindAllStringSubmatch(hexDump, -1)
if hexDumpRowValues == nil || len(hexDumpRowValues) != strings.Count(hexDump, "\n") {
message := "could not parse hexdump"
gologger.Warning().Msgf(message)
gologger.Warning().Msg(message)
return HighlightableHexDump{}, errors.New(message)
}
+13 -2
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"net"
"net/url"
"sync"
"github.com/go-sql-driver/mysql"
"github.com/pkg/errors"
@@ -19,9 +20,17 @@ import (
// Dialer is a shared fastdialer instance for host DNS resolution
var (
Dialer *fastdialer.Dialer
muDialer sync.RWMutex
Dialer *fastdialer.Dialer
)
func GetDialer() *fastdialer.Dialer {
muDialer.RLock()
defer muDialer.RUnlock()
return Dialer
}
func ShouldInit() bool {
return Dialer == nil
}
@@ -210,10 +219,12 @@ func interfaceAddresses(interfaceName string) ([]net.Addr, error) {
// Close closes the global shared fastdialer
func Close() {
muDialer.Lock()
defer muDialer.Unlock()
if Dialer != nil {
Dialer.Close()
Dialer = nil
}
Dialer = nil
StopActiveMemGuardian()
}
+3 -3
View File
@@ -106,7 +106,7 @@ func (request *Request) ExecuteWithResults(input *contextargs.Context, metadata,
}
func (request *Request) execute(input *contextargs.Context, domain string, metadata, previous output.InternalEvent, vars map[string]interface{}, callback protocols.OutputEventCallback) error {
var err error
if vardump.EnableVarDump {
gologger.Debug().Msgf("DNS Protocol request variables: \n%s\n", vardump.DumpVariables(vars))
}
@@ -142,7 +142,7 @@ func (request *Request) execute(input *contextargs.Context, domain string, metad
if request.options.Options.Debug || request.options.Options.DebugRequests || request.options.Options.StoreResponse {
msg := fmt.Sprintf("[%s] Dumped DNS request for %s", request.options.TemplateID, question)
if request.options.Options.Debug || request.options.Options.DebugRequests {
gologger.Info().Str("domain", domain).Msgf(msg)
gologger.Info().Str("domain", domain).Msg(msg)
gologger.Print().Msgf("%s", requestString)
}
if request.options.Options.StoreResponse {
@@ -199,7 +199,7 @@ func (request *Request) execute(input *contextargs.Context, domain string, metad
}
callback(event)
return nil
return err
}
func (request *Request) parseDNSInput(host string) (string, error) {
@@ -6,11 +6,15 @@ import (
"strings"
"github.com/invopop/jsonschema"
mapsutil "github.com/projectdiscovery/utils/maps"
)
// ActionType defines the action type for a browser action
type ActionType int8
// ActionData stores the action output data
type ActionData = mapsutil.Map[string, any]
// Types to be executed by the user.
// name:ActionType
const (
@@ -68,6 +72,9 @@ const (
// ActionWaitEvent waits for a specific event.
// name:waitevent
ActionWaitEvent
// ActionWaitDialog waits for JavaScript dialog (alert, confirm, prompt, or onbeforeunload).
// name:dialog
ActionWaitDialog
// ActionKeyboard performs a keyboard action event on a page.
// name:keyboard
ActionKeyboard
@@ -104,6 +111,7 @@ var ActionStringToAction = map[string]ActionType{
"deleteheader": ActionDeleteHeader,
"setbody": ActionSetBody,
"waitevent": ActionWaitEvent,
"waitdialog": ActionWaitDialog,
"keyboard": ActionKeyboard,
"debug": ActionDebug,
"sleep": ActionSleep,
@@ -130,6 +138,7 @@ var ActionToActionString = map[ActionType]string{
ActionDeleteHeader: "deleteheader",
ActionSetBody: "setbody",
ActionWaitEvent: "waitevent",
ActionWaitDialog: "waitdialog",
ActionKeyboard: "keyboard",
ActionDebug: "debug",
ActionSleep: "sleep",
+1 -1
View File
@@ -45,7 +45,7 @@ type Options struct {
}
// Run runs a list of actions by creating a new page in the browser.
func (i *Instance) Run(input *contextargs.Context, actions []*Action, payloads map[string]interface{}, options *Options) (map[string]string, *Page, error) {
func (i *Instance) Run(input *contextargs.Context, actions []*Action, payloads map[string]interface{}, options *Options) (ActionData, *Page, error) {
page, err := i.engine.Page(proto.TargetCreateTarget{})
if err != nil {
return nil, nil, err
+69 -24
View File
@@ -14,6 +14,7 @@ import (
"github.com/go-rod/rod/lib/input"
"github.com/go-rod/rod/lib/proto"
"github.com/go-rod/rod/lib/utils"
"github.com/kitabisa/go-ci"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/contextargs"
@@ -47,14 +48,19 @@ const (
)
// ExecuteActions executes a list of actions on a page.
func (p *Page) ExecuteActions(input *contextargs.Context, actions []*Action, variables map[string]interface{}) (outData map[string]string, err error) {
outData = make(map[string]string)
func (p *Page) ExecuteActions(input *contextargs.Context, actions []*Action, variables map[string]interface{}) (outData ActionData, err error) {
outData = make(ActionData)
// waitFuncs are function that needs to be executed after navigation
// typically used for waitEvent
waitFuncs := make([]func() error, 0)
// avoid any future panics caused due to go-rod library
// TODO(dwisiswant0): remove this once we get the RCA.
defer func() {
if ci.IsCI() {
return
}
if r := recover(); r != nil {
err = errorutil.New("panic on headless action: %v", r)
}
@@ -100,6 +106,8 @@ func (p *Page) ExecuteActions(input *contextargs.Context, actions []*Action, var
if waitFunc != nil {
waitFuncs = append(waitFuncs, waitFunc)
}
case ActionWaitDialog:
err = p.HandleDialog(act, outData)
case ActionFilesInput:
if p.options.Options.AllowLocalFileAccess {
err = p.FilesInput(act, outData)
@@ -142,7 +150,7 @@ type rule struct {
}
// WaitVisible waits until an element appears.
func (p *Page) WaitVisible(act *Action, out map[string]string) error {
func (p *Page) WaitVisible(act *Action, out ActionData) error {
timeout, err := getTimeout(p, act)
if err != nil {
return errors.Wrap(err, "Wrong timeout given")
@@ -217,7 +225,7 @@ func geTimeParameter(p *Page, act *Action, parameterName string, defaultValue ti
}
// ActionAddHeader executes a AddHeader action.
func (p *Page) ActionAddHeader(act *Action, out map[string]string) error {
func (p *Page) ActionAddHeader(act *Action, out ActionData) error {
in := p.getActionArgWithDefaultValues(act, "part")
args := make(map[string]string)
@@ -228,7 +236,7 @@ func (p *Page) ActionAddHeader(act *Action, out map[string]string) error {
}
// ActionSetHeader executes a SetHeader action.
func (p *Page) ActionSetHeader(act *Action, out map[string]string) error {
func (p *Page) ActionSetHeader(act *Action, out ActionData) error {
in := p.getActionArgWithDefaultValues(act, "part")
args := make(map[string]string)
@@ -239,7 +247,7 @@ func (p *Page) ActionSetHeader(act *Action, out map[string]string) error {
}
// ActionDeleteHeader executes a DeleteHeader action.
func (p *Page) ActionDeleteHeader(act *Action, out map[string]string) error {
func (p *Page) ActionDeleteHeader(act *Action, out ActionData) error {
in := p.getActionArgWithDefaultValues(act, "part")
args := make(map[string]string)
@@ -249,7 +257,7 @@ func (p *Page) ActionDeleteHeader(act *Action, out map[string]string) error {
}
// ActionSetBody executes a SetBody action.
func (p *Page) ActionSetBody(act *Action, out map[string]string) error {
func (p *Page) ActionSetBody(act *Action, out ActionData) error {
in := p.getActionArgWithDefaultValues(act, "part")
args := make(map[string]string)
@@ -259,7 +267,7 @@ func (p *Page) ActionSetBody(act *Action, out map[string]string) error {
}
// ActionSetMethod executes an SetMethod action.
func (p *Page) ActionSetMethod(act *Action, out map[string]string) error {
func (p *Page) ActionSetMethod(act *Action, out ActionData) error {
in := p.getActionArgWithDefaultValues(act, "part")
args := make(map[string]string)
@@ -269,7 +277,7 @@ func (p *Page) ActionSetMethod(act *Action, out map[string]string) error {
}
// NavigateURL executes an ActionLoadURL actions loading a URL for the page.
func (p *Page) NavigateURL(action *Action, out map[string]string, allvars map[string]interface{}) error {
func (p *Page) NavigateURL(action *Action, out ActionData, allvars map[string]interface{}) error {
// input <- is input url from cli
// target <- is the url from template (ex: {{BaseURL}}/test)
input, err := urlutil.Parse(p.input.MetaInput.Input)
@@ -325,7 +333,7 @@ func (p *Page) NavigateURL(action *Action, out map[string]string, allvars map[st
}
// RunScript runs a script on the loaded page
func (p *Page) RunScript(action *Action, out map[string]string) error {
func (p *Page) RunScript(action *Action, out ActionData) error {
code := p.getActionArgWithDefaultValues(action, "code")
if code == "" {
return errinvalidArguments
@@ -346,7 +354,7 @@ func (p *Page) RunScript(action *Action, out map[string]string) error {
}
// ClickElement executes click actions for an element.
func (p *Page) ClickElement(act *Action, out map[string]string) error {
func (p *Page) ClickElement(act *Action, out ActionData) error {
element, err := p.pageElementBy(act.Data)
if err != nil {
return errors.Wrap(err, errCouldNotGetElement)
@@ -361,12 +369,12 @@ func (p *Page) ClickElement(act *Action, out map[string]string) error {
}
// KeyboardAction executes a keyboard action on the page.
func (p *Page) KeyboardAction(act *Action, out map[string]string) error {
func (p *Page) KeyboardAction(act *Action, out ActionData) error {
return p.page.Keyboard.Type([]input.Key(p.getActionArgWithDefaultValues(act, "keys"))...)
}
// RightClickElement executes right click actions for an element.
func (p *Page) RightClickElement(act *Action, out map[string]string) error {
func (p *Page) RightClickElement(act *Action, out ActionData) error {
element, err := p.pageElementBy(act.Data)
if err != nil {
return errors.Wrap(err, errCouldNotGetElement)
@@ -381,7 +389,7 @@ func (p *Page) RightClickElement(act *Action, out map[string]string) error {
}
// Screenshot executes screenshot action on a page
func (p *Page) Screenshot(act *Action, out map[string]string) error {
func (p *Page) Screenshot(act *Action, out ActionData) error {
to := p.getActionArgWithDefaultValues(act, "to")
if to == "" {
to = ksuid.New().String()
@@ -444,7 +452,7 @@ func (p *Page) Screenshot(act *Action, out map[string]string) error {
}
// InputElement executes input element actions for an element.
func (p *Page) InputElement(act *Action, out map[string]string) error {
func (p *Page) InputElement(act *Action, out ActionData) error {
value := p.getActionArgWithDefaultValues(act, "value")
if value == "" {
return errinvalidArguments
@@ -463,7 +471,7 @@ func (p *Page) InputElement(act *Action, out map[string]string) error {
}
// TimeInputElement executes time input on an element
func (p *Page) TimeInputElement(act *Action, out map[string]string) error {
func (p *Page) TimeInputElement(act *Action, out ActionData) error {
value := p.getActionArgWithDefaultValues(act, "value")
if value == "" {
return errinvalidArguments
@@ -486,7 +494,7 @@ func (p *Page) TimeInputElement(act *Action, out map[string]string) error {
}
// SelectInputElement executes select input statement action on a element
func (p *Page) SelectInputElement(act *Action, out map[string]string) error {
func (p *Page) SelectInputElement(act *Action, out ActionData) error {
value := p.getActionArgWithDefaultValues(act, "value")
if value == "" {
return errinvalidArguments
@@ -511,7 +519,7 @@ func (p *Page) SelectInputElement(act *Action, out map[string]string) error {
}
// WaitLoad waits for the page to load
func (p *Page) WaitLoad(act *Action, out map[string]string) error {
func (p *Page) WaitLoad(act *Action, out ActionData) error {
p.page.Timeout(2 * time.Second).WaitNavigation(proto.PageLifecycleEventNameFirstMeaningfulPaint)()
// Wait for the window.onload event and also wait for the network requests
@@ -525,7 +533,7 @@ func (p *Page) WaitLoad(act *Action, out map[string]string) error {
}
// GetResource gets a resource from an element from page.
func (p *Page) GetResource(act *Action, out map[string]string) error {
func (p *Page) GetResource(act *Action, out ActionData) error {
element, err := p.pageElementBy(act.Data)
if err != nil {
return errors.Wrap(err, errCouldNotGetElement)
@@ -541,7 +549,7 @@ func (p *Page) GetResource(act *Action, out map[string]string) error {
}
// FilesInput acts with a file input element on page
func (p *Page) FilesInput(act *Action, out map[string]string) error {
func (p *Page) FilesInput(act *Action, out ActionData) error {
element, err := p.pageElementBy(act.Data)
if err != nil {
return errors.Wrap(err, errCouldNotGetElement)
@@ -558,7 +566,7 @@ func (p *Page) FilesInput(act *Action, out map[string]string) error {
}
// ExtractElement extracts from an element on the page.
func (p *Page) ExtractElement(act *Action, out map[string]string) error {
func (p *Page) ExtractElement(act *Action, out ActionData) error {
element, err := p.pageElementBy(act.Data)
if err != nil {
return errors.Wrap(err, errCouldNotGetElement)
@@ -592,7 +600,7 @@ func (p *Page) ExtractElement(act *Action, out map[string]string) error {
}
// WaitEvent waits for an event to happen on the page.
func (p *Page) WaitEvent(act *Action, out map[string]string) (func() error, error) {
func (p *Page) WaitEvent(act *Action, out ActionData) (func() error, error) {
event := p.getActionArgWithDefaultValues(act, "event")
if event == "" {
return nil, errors.New("event not recognized")
@@ -630,6 +638,43 @@ func (p *Page) WaitEvent(act *Action, out map[string]string) (func() error, erro
return waitFunc, nil
}
// HandleDialog handles JavaScript dialog (alert, confirm, prompt, or onbeforeunload).
func (p *Page) HandleDialog(act *Action, out ActionData) error {
maxDuration := 10 * time.Second
if dur := p.getActionArgWithDefaultValues(act, "max-duration"); dur != "" {
var err error
maxDuration, err = time.ParseDuration(dur)
if err != nil {
return errorutil.NewWithErr(err).Msgf("could not parse max-duration")
}
}
ctx, cancel := context.WithTimeout(context.Background(), maxDuration)
defer cancel()
wait, handle := p.page.HandleDialog()
fn := func() (*proto.PageJavascriptDialogOpening, error) {
dialog := wait()
err := handle(&proto.PageHandleJavaScriptDialog{
Accept: true,
PromptText: "",
})
return dialog, err
}
dialog, err := contextutil.ExecFuncWithTwoReturns(ctx, fn)
if err == nil && act.Name != "" {
out[act.Name] = true
out[act.Name+"_type"] = string(dialog.Type)
out[act.Name+"_message"] = dialog.Message
}
return nil
}
// pageElementBy returns a page element from a variety of inputs.
//
// Supported values for by: r -> selector & regex, x -> xpath, js -> eval js,
@@ -664,14 +709,14 @@ func (p *Page) pageElementBy(data map[string]string) (*rod.Element, error) {
}
// DebugAction enables debug action on a page.
func (p *Page) DebugAction(act *Action, out map[string]string) error {
func (p *Page) DebugAction(act *Action, out ActionData) error {
p.instance.browser.engine.SlowMotion(5 * time.Second)
p.instance.browser.engine.Trace(true)
return nil
}
// SleepAction sleeps on the page for a specified duration
func (p *Page) SleepAction(act *Action, out map[string]string) error {
func (p *Page) SleepAction(act *Action, out ActionData) error {
seconds := act.Data["duration"]
if seconds == "" {
seconds = "5"
@@ -38,7 +38,7 @@ func TestActionNavigate(t *testing.T) {
actions := []*Action{{ActionType: ActionTypeHolder{ActionType: ActionNavigate}, Data: map[string]string{"url": "{{BaseURL}}"}}, {ActionType: ActionTypeHolder{ActionType: ActionWaitLoad}}}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nilf(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
})
@@ -63,7 +63,7 @@ func TestActionScript(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionScript}, Name: "test", Data: map[string]string{"code": "() => window.test"}},
}
testHeadlessSimpleResponse(t, response, actions, timeout, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, timeout, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
require.Equal(t, "some-data", out["test"], "could not run js and get results correctly")
@@ -77,7 +77,7 @@ func TestActionScript(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionWaitLoad}},
{ActionType: ActionTypeHolder{ActionType: ActionScript}, Name: "test", Data: map[string]string{"code": "() => window.test"}},
}
testHeadlessSimpleResponse(t, response, actions, timeout, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, timeout, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
require.Equal(t, "some-data", out["test"], "could not run js and get results correctly with js hook")
@@ -101,7 +101,7 @@ func TestActionClick(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionClick}, Data: map[string]string{"selector": "button"}}, // Use css selector for clicking
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
el := page.Page().MustElement("button")
@@ -134,7 +134,7 @@ func TestActionRightClick(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionRightClick}, Data: map[string]string{"selector": "button"}}, // Use css selector for clicking
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
el := page.Page().MustElement("button")
@@ -159,7 +159,7 @@ func TestActionTextInput(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionTextInput}, Data: map[string]string{"selector": "input", "value": "test"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
el := page.Page().MustElement("input")
@@ -182,7 +182,7 @@ func TestActionHeadersChange(t *testing.T) {
}
}
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out map[string]string) {
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "found", strings.ToLower(strings.TrimSpace(page.Page().MustElement("html").MustText())), "could not set header correctly")
})
@@ -205,7 +205,7 @@ func TestActionScreenshot(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionScreenshot}, Data: map[string]string{"to": filePath}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
_ = page.Page()
@@ -233,7 +233,7 @@ func TestActionScreenshotToDir(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionScreenshot}, Data: map[string]string{"to": filePath, "mkdir": "true"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
_ = page.Page()
@@ -260,7 +260,7 @@ func TestActionTimeInput(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionTimeInput}, Data: map[string]string{"selector": "input", "value": "2006-01-02T15:04:05Z"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
el := page.Page().MustElement("input")
@@ -288,7 +288,7 @@ func TestActionSelectInput(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionSelectInput}, Data: map[string]string{"by": "x", "xpath": "//select[@id='test']", "value": "Test2", "selected": "true"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
el := page.Page().MustElement("select")
require.Equal(t, "Test2", el.MustText(), "could not get input change value")
@@ -311,7 +311,7 @@ func TestActionFilesInput(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionFilesInput}, Data: map[string]string{"selector": "input", "value": "test1.pdf"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Nuclei Test Page", page.Page().MustInfo().Title, "could not navigate correctly")
el := page.Page().MustElement("input")
@@ -337,7 +337,7 @@ func TestActionFilesInputNegative(t *testing.T) {
}
t.Setenv("LOCAL_FILE_ACCESS", "false")
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.ErrorContains(t, err, ErrLFAccessDenied.Error(), "got file access when -lfa is false")
})
}
@@ -359,7 +359,7 @@ func TestActionWaitLoad(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionWaitLoad}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
el := page.Page().MustElement("button")
style, attributeErr := el.Attribute("style")
@@ -384,9 +384,12 @@ func TestActionGetResource(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionGetResource}, Data: map[string]string{"by": "x", "xpath": "//img[@id='test']"}, Name: "src"},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, len(out["src"]), 121808, "could not find resource")
src, ok := out["src"].(string)
require.True(t, ok, "could not assert src to string")
require.Equal(t, len(src), 121808, "could not find resource")
})
}
@@ -404,7 +407,7 @@ func TestActionExtract(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionExtract}, Data: map[string]string{"by": "x", "xpath": "//button[@id='test']"}, Name: "extract"},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "Wait for me!", out["extract"], "could not extract text")
})
@@ -423,7 +426,7 @@ func TestActionSetMethod(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionSetMethod}, Data: map[string]string{"part": "x", "method": "SET"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "SET", page.rules[0].Args["method"], "could not find resource")
})
@@ -442,7 +445,7 @@ func TestActionAddHeader(t *testing.T) {
}
}
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out map[string]string) {
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "found", strings.ToLower(strings.TrimSpace(page.Page().MustElement("html").MustText())), "could not set header correctly")
})
@@ -463,7 +466,7 @@ func TestActionDeleteHeader(t *testing.T) {
}
}
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out map[string]string) {
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "header deleted", strings.ToLower(strings.TrimSpace(page.Page().MustElement("html").MustText())), "could not delete header correctly")
})
@@ -481,7 +484,7 @@ func TestActionSetBody(t *testing.T) {
_, _ = fmt.Fprintln(w, string(body))
}
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out map[string]string) {
testHeadless(t, actions, 20*time.Second, handler, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.Equal(t, "hello", strings.ToLower(strings.TrimSpace(page.Page().MustElement("html").MustText())), "could not set header correctly")
})
@@ -505,7 +508,7 @@ func TestActionKeyboard(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionKeyboard}, Data: map[string]string{"keys": "Test2"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
el := page.Page().MustElement("input")
require.Equal(t, "Test2", el.MustText(), "could not get input change value")
@@ -529,7 +532,7 @@ func TestActionSleep(t *testing.T) {
{ActionType: ActionTypeHolder{ActionType: ActionSleep}, Data: map[string]string{"duration": "2"}},
}
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 20*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
require.True(t, page.Page().MustElement("button").MustVisible(), "could not get button")
})
@@ -553,7 +556,7 @@ func TestActionWaitVisible(t *testing.T) {
}
t.Run("wait for an element being visible", func(t *testing.T) {
testHeadlessSimpleResponse(t, response, actions, 2*time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, 2*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
page.Page().MustElement("button").MustVisible()
@@ -562,21 +565,82 @@ func TestActionWaitVisible(t *testing.T) {
t.Run("timeout because of element not visible", func(t *testing.T) {
// increased timeout from time.Second/2 to time.Second due to random fails (probably due to overhead and system)
testHeadlessSimpleResponse(t, response, actions, time.Second, func(page *Page, err error, out map[string]string) {
testHeadlessSimpleResponse(t, response, actions, time.Second, func(page *Page, err error, out ActionData) {
require.Error(t, err)
require.Contains(t, err.Error(), "Element did not appear in the given amount of time")
})
})
}
func testHeadlessSimpleResponse(t *testing.T, response string, actions []*Action, timeout time.Duration, assert func(page *Page, pageErr error, out map[string]string)) {
func TestActionWaitDialog(t *testing.T) {
response := `<html>
<head>
<title>Nuclei Test Page</title>
</head>
<body>
<script type="text/javascript">
const urlParams = new URLSearchParams(window.location.search);
const scriptContent = urlParams.get('script');
if (scriptContent) {
const scriptElement = document.createElement('script');
scriptElement.textContent = scriptContent;
document.body.appendChild(scriptElement);
}
</script>
</body>
</html>`
t.Run("Triggered", func(t *testing.T) {
actions := []*Action{
{
ActionType: ActionTypeHolder{ActionType: ActionNavigate},
Data: map[string]string{"url": "{{BaseURL}}/?script=alert%281%29"},
},
{
ActionType: ActionTypeHolder{ActionType: ActionWaitDialog},
Name: "test",
},
}
testHeadlessSimpleResponse(t, response, actions, 1*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
test, ok := out["test"].(bool)
require.True(t, ok, "could not assert test to bool")
require.True(t, test, "could not find test")
})
})
t.Run("Invalid", func(t *testing.T) {
actions := []*Action{
{
ActionType: ActionTypeHolder{ActionType: ActionNavigate},
Data: map[string]string{"url": "{{BaseURL}}/?script=foo"},
},
{
ActionType: ActionTypeHolder{ActionType: ActionWaitDialog},
Name: "test",
},
}
testHeadlessSimpleResponse(t, response, actions, 1*time.Second, func(page *Page, err error, out ActionData) {
require.Nil(t, err, "could not run page actions")
_, ok := out["test"].(bool)
require.False(t, ok, "output assertion is success")
})
})
}
func testHeadlessSimpleResponse(t *testing.T, response string, actions []*Action, timeout time.Duration, assert func(page *Page, pageErr error, out ActionData)) {
t.Helper()
testHeadless(t, actions, timeout, func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintln(w, response)
}, assert)
}
func testHeadless(t *testing.T, actions []*Action, timeout time.Duration, handler func(w http.ResponseWriter, r *http.Request), assert func(page *Page, pageErr error, extractedData map[string]string)) {
func testHeadless(t *testing.T, actions []*Action, timeout time.Duration, handler func(w http.ResponseWriter, r *http.Request), assert func(page *Page, pageErr error, extractedData ActionData)) {
t.Helper()
lfa := getBoolFromEnv("LOCAL_FILE_ACCESS", true)
+8 -2
View File
@@ -174,7 +174,7 @@ func (request *Request) executeRequestWithPayloads(input *contextargs.Context, p
reqBuilder.WriteString("\t" + actStepStr + "\n")
}
}
gologger.Debug().Msgf(reqBuilder.String())
gologger.Debug().Msg(reqBuilder.String())
}
var responseBody string
@@ -183,7 +183,13 @@ func (request *Request) executeRequestWithPayloads(input *contextargs.Context, p
responseBody, _ = html.HTML()
}
outputEvent := request.responseToDSLMap(responseBody, out["header"], out["status_code"], reqBuilder.String(), input.MetaInput.Input, navigatedURL, page.DumpHistory())
header := out.GetOrDefault("header", "").(string)
// NOTE(dwisiswant0): `status_code` key should be an integer type.
// Ref: https://github.com/projectdiscovery/nuclei/pull/5545#discussion_r1721291013
statusCode := out.GetOrDefault("status_code", "").(string)
outputEvent := request.responseToDSLMap(responseBody, header, statusCode, reqBuilder.String(), input.MetaInput.Input, navigatedURL, page.DumpHistory())
// add response fields to template context and merge templatectx variables to output event
request.options.AddTemplateVars(input.MetaInput, request.Type(), request.ID, outputEvent)
if request.options.HasTemplateCtx(input.MetaInput) {
+8 -8
View File
@@ -90,9 +90,9 @@ func (g *generatedRequest) ApplyAuth(provider authprovider.AuthProvider) {
return
}
if g.request != nil {
auth := provider.LookupURLX(g.request.URL)
if auth != nil {
auth.ApplyOnRR(g.request)
authStrategies := provider.LookupURLX(g.request.URL)
for _, strategy := range authStrategies {
strategy.ApplyOnRR(g.request)
}
}
if g.rawRequest != nil {
@@ -101,11 +101,11 @@ func (g *generatedRequest) ApplyAuth(provider authprovider.AuthProvider) {
gologger.Warning().Msgf("[authprovider] Could not parse URL %s: %s\n", g.rawRequest.FullURL, err)
return
}
auth := provider.LookupURLX(parsed)
if auth != nil {
// here we need to apply it custom because we don't have a standard/official
// rawhttp request format ( which we probably should have )
g.rawRequest.ApplyAuthStrategy(auth)
authStrategies := provider.LookupURLX(parsed)
// here we need to apply it custom because we don't have a standard/official
// rawhttp request format ( which we probably should have )
for _, strategy := range authStrategies {
g.rawRequest.ApplyAuthStrategy(strategy)
}
}
}
+4
View File
@@ -147,6 +147,10 @@ type Request struct {
// - "AWS"
Signature SignatureTypeHolder `yaml:"signature,omitempty" json:"signature,omitempty" jsonschema:"title=signature is the http request signature method,description=Signature is the HTTP Request signature Method,enum=AWS"`
// description: |
// SkipSecretFile skips the authentication or authorization configured in the secret file.
SkipSecretFile bool `yaml:"skip-secret-file,omitempty" json:"skip-secret-file,omitempty" jsonschema:"title=bypass secret file,description=Skips the authentication or authorization configured in the secret file"`
// description: |
// CookieReuse is an optional setting that enables cookie reuse for
// all requests defined in raw section.
@@ -29,6 +29,7 @@ import (
var (
rawHttpClient *rawhttp.Client
rawHttpClientOnce sync.Once
forceMaxRedirects int
normalClient *retryablehttp.Client
clientPool *mapsutil.SyncLockMap[string, *retryablehttp.Client]
@@ -102,6 +103,22 @@ type Configuration struct {
ResponseHeaderTimeout time.Duration
}
func (c *Configuration) Clone() *Configuration {
clone := *c
if c.Connection != nil {
cloneConnection := &ConnectionConfiguration{
DisableKeepAlive: c.Connection.DisableKeepAlive,
}
if c.Connection.HasCookieJar() {
cookiejar := *c.Connection.GetCookieJar()
cloneConnection.SetCookieJar(&cookiejar)
}
clone.Connection = cloneConnection
}
return &clone
}
// Hash returns the hash of the configuration to allow client pooling
func (c *Configuration) Hash() string {
builder := &strings.Builder{}
@@ -131,7 +148,7 @@ func (c *Configuration) HasStandardOptions() bool {
// GetRawHTTP returns the rawhttp request client
func GetRawHTTP(options *protocols.ExecutorOptions) *rawhttp.Client {
if rawHttpClient == nil {
rawHttpClientOnce.Do(func() {
rawHttpOptions := rawhttp.DefaultOptions
if types.ProxyURL != "" {
rawHttpOptions.Proxy = types.ProxyURL
@@ -142,7 +159,7 @@ func GetRawHTTP(options *protocols.ExecutorOptions) *rawhttp.Client {
}
rawHttpOptions.Timeout = options.Options.GetTimeouts().HttpTimeout
rawHttpClient = rawhttp.NewClient(rawHttpOptions)
}
})
return rawHttpClient
}
@@ -233,7 +250,7 @@ func wrappedGet(options *types.Options, configuration *Configuration) (*retryabl
transport := &http.Transport{
ForceAttemptHTTP2: options.ForceAttemptHTTP2,
DialContext: protocolstate.Dialer.Dial,
DialContext: protocolstate.GetDialer().Dial,
DialTLSContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
if options.TlsImpersonate {
return protocolstate.Dialer.DialTLSWithConfigImpersonate(ctx, network, addr, tlsConfig, impersonate.Random, nil)
@@ -241,7 +258,7 @@ func wrappedGet(options *types.Options, configuration *Configuration) (*retryabl
if options.HasClientCertificates() || options.ForceAttemptHTTP2 {
return protocolstate.Dialer.DialTLSWithConfig(ctx, network, addr, tlsConfig)
}
return protocolstate.Dialer.DialTLS(ctx, network, addr)
return protocolstate.GetDialer().DialTLS(ctx, network, addr)
},
MaxIdleConns: maxIdleConns,
MaxIdleConnsPerHost: maxIdleConnsPerHost,
+9 -4
View File
@@ -689,7 +689,7 @@ func (request *Request) executeRequest(input *contextargs.Context, generatedRequ
}
// === apply auth strategies ===
if generatedRequest.request != nil {
if generatedRequest.request != nil && !request.SkipSecretFile {
generatedRequest.ApplyAuth(request.options.AuthProvider)
}
@@ -770,7 +770,7 @@ func (request *Request) executeRequest(input *contextargs.Context, generatedRequ
// check for cookie related configuration
if input.CookieJar != nil {
connConfiguration := request.connConfiguration
connConfiguration := request.connConfiguration.Clone()
connConfiguration.Connection.SetCookieJar(input.CookieJar)
modifiedConfig = connConfiguration
}
@@ -778,7 +778,8 @@ func (request *Request) executeRequest(input *contextargs.Context, generatedRequ
updatedTimeout, ok := generatedRequest.request.Context().Value(httpclientpool.WithCustomTimeout{}).(httpclientpool.WithCustomTimeout)
if ok {
if modifiedConfig == nil {
modifiedConfig = request.connConfiguration
connConfiguration := request.connConfiguration.Clone()
modifiedConfig = connConfiguration
}
modifiedConfig.ResponseHeaderTimeout = updatedTimeout.Timeout
}
@@ -941,7 +942,11 @@ func (request *Request) executeRequest(input *contextargs.Context, generatedRequ
if input.MetaInput.CustomIP != "" {
outputEvent["ip"] = input.MetaInput.CustomIP
} else {
outputEvent["ip"] = protocolstate.Dialer.GetDialedIP(hostname)
dialer := protocolstate.GetDialer()
if dialer != nil {
outputEvent["ip"] = dialer.GetDialedIP(hostname)
}
// try getting cname
request.addCNameIfAvailable(hostname, outputEvent)
}
+8 -6
View File
@@ -18,6 +18,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/contextargs"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/vardump"
protocolutils "github.com/projectdiscovery/nuclei/v3/pkg/protocols/utils"
@@ -112,6 +113,7 @@ func (request *Request) executeFuzzingRule(input *contextargs.Context, previous
// executeAllFuzzingRules executes all fuzzing rules defined in template for a given base request
func (request *Request) executeAllFuzzingRules(input *contextargs.Context, values map[string]interface{}, baseRequest *retryablehttp.Request, callback protocols.OutputEventCallback) error {
applicable := false
values = generators.MergeMaps(request.filterDataMap(input), values)
for _, rule := range request.Fuzzing {
select {
case <-input.Context().Done():
@@ -234,7 +236,12 @@ func (request *Request) ShouldFuzzTarget(input *contextargs.Context) bool {
}
status := []bool{}
for index, filter := range request.FuzzPreCondition {
isMatch, _ := request.Match(request.filterDataMap(input), filter)
dataMap := request.filterDataMap(input)
// dump if svd is enabled
if request.options.Options.ShowVarDump {
gologger.Debug().Msgf("Fuzz Filter Variables: \n%s\n", vardump.DumpVariables(dataMap))
}
isMatch, _ := request.Match(dataMap, filter)
status = append(status, isMatch)
if request.options.Options.MatcherStatus {
gologger.Debug().Msgf("[%s] [%s] Filter => %s : %v", input.MetaInput.Target(), request.options.TemplateID, operators.GetMatcherName(filter, index), isMatch)
@@ -295,10 +302,5 @@ func (request *Request) filterDataMap(input *contextargs.Context) map[string]int
// add default method value
m["method"] = http.MethodGet
}
// dump if svd is enabled
if request.options.Options.ShowVarDump {
gologger.Debug().Msgf("Fuzz Filter Variables: \n%s\n", vardump.DumpVariables(m))
}
return m
}
+2 -2
View File
@@ -155,14 +155,14 @@ func (request *Request) executeOnTarget(input *contextargs.Context, visited maps
}
visited.Set(actualAddress, struct{}{})
if err := request.executeAddress(variables, actualAddress, address, input, kv.tls, previous, callback); err != nil {
if err = request.executeAddress(variables, actualAddress, address, input, kv.tls, previous, callback); err != nil {
outputEvent := request.responseToDSLMap("", "", "", address, "")
callback(&output.InternalWrappedEvent{InternalEvent: outputEvent})
gologger.Warning().Msgf("[%v] Could not make network request for (%s) : %s\n", request.options.TemplateID, actualAddress, err)
continue
}
}
return nil
return err
}
// executeAddress executes the request for an address
+1 -1
View File
@@ -86,7 +86,7 @@ func TestNetworkExecuteWithResults(t *testing.T) {
err := request.ExecuteWithResults(ctxArgs, metadata, previous, func(event *output.InternalWrappedEvent) {
finalEvent = event
})
require.Nil(t, err, "could not execute network request")
require.NotNil(t, err, "could not execute network request")
})
require.Nil(t, finalEvent.Results, "could not get event output from request")
+3
View File
@@ -12,6 +12,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/github"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/gitlab"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/jira"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/linear"
"github.com/projectdiscovery/retryablehttp-go"
)
@@ -29,6 +30,8 @@ type Options struct {
Gitea *gitea.Options `yaml:"gitea"`
// Jira contains configuration options for Jira Issue Tracker
Jira *jira.Options `yaml:"jira"`
// Linear contains configuration options for Linear Issue Tracker
Linear *linear.Options `yaml:"linear"`
// MarkdownExporter contains configuration options for Markdown Exporter Module
MarkdownExporter *markdown.Options `yaml:"markdown"`
// SarifExporter contains configuration options for Sarif Exporter Module
+12 -1
View File
@@ -28,6 +28,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/github"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/gitlab"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/jira"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/linear"
errorutil "github.com/projectdiscovery/utils/errors"
fileutil "github.com/projectdiscovery/utils/file"
)
@@ -112,6 +113,15 @@ func New(options *Options, db string, doNotDedupe bool) (Client, error) {
}
client.trackers = append(client.trackers, tracker)
}
if options.Linear != nil {
options.Linear.HttpClient = options.HttpClient
options.Linear.OmitRaw = options.OmitRaw
tracker, err := linear.New(options.Linear)
if err != nil {
return nil, errorutil.NewWithErr(err).Wrap(ErrReportingClientCreation)
}
client.trackers = append(client.trackers, tracker)
}
if options.MarkdownExporter != nil {
exporter, err := markdown.New(options.MarkdownExporter)
if err != nil {
@@ -195,6 +205,7 @@ func CreateConfigIfNotExists() error {
GitLab: &gitlab.Options{},
Gitea: &gitea.Options{},
Jira: &jira.Options{},
Linear: &linear.Options{},
MarkdownExporter: &markdown.Options{},
SarifExporter: &sarif.Options{},
ElasticsearchExporter: &es.Options{},
@@ -240,7 +251,7 @@ func (c *ReportingClient) Close() {
if failed > 0 {
msgBuilder.WriteString(fmt.Sprintf(", %d failed", failed))
}
gologger.Info().Msgf(msgBuilder.String())
gologger.Info().Msgf("%v", msgBuilder.String())
}
}
}
+1 -1
View File
@@ -78,7 +78,7 @@ func (i *Integration) CreateIssue(event *output.ResultEvent) (*filters.CreateIss
if label := i.options.IssueLabel; label != "" {
labels = append(labels, label)
}
customLabels := gitlab.Labels(labels)
customLabels := gitlab.LabelOptions(labels)
assigneeIDs := []int{i.userID}
if i.options.DuplicateIssueCheck {
searchIn := "title"
+12 -1
View File
@@ -70,9 +70,13 @@ type Options struct {
// Token is the token for jira instance.
Token string `yaml:"token" json:"token" validate:"required"`
// ProjectName is the name of the project.
ProjectName string `yaml:"project-name" json:"project_name" validate:"required"`
ProjectName string `yaml:"project-name" json:"project_name"`
// ProjectID is the ID of the project (optional)
ProjectID string `yaml:"project-id" json:"project_id"`
// IssueType (optional) is the name of the created issue type
IssueType string `yaml:"issue-type" json:"issue_type"`
// IssueTypeID (optional) is the ID of the created issue type
IssueTypeID string `yaml:"issue-type-id" json:"issue_type_id"`
// SeverityAsLabel (optional) sends the severity as the label of the created
// issue.
SeverityAsLabel bool `yaml:"severity-as-label" json:"severity_as_label"`
@@ -183,6 +187,7 @@ func (i *Integration) CreateNewIssue(event *output.ResultEvent) (*filters.Create
Project: jira.Project{Key: i.options.ProjectName},
Summary: summary,
}
// On-prem version of Jira server does not use AccountID
if !i.options.Cloud {
fields = &jira.IssueFields{
@@ -195,6 +200,12 @@ func (i *Integration) CreateNewIssue(event *output.ResultEvent) (*filters.Create
Unknowns: customFields,
}
}
if i.options.IssueTypeID != "" {
fields.Type = jira.IssueType{ID: i.options.IssueTypeID}
}
if i.options.ProjectID != "" {
fields.Project = jira.Project{ID: i.options.ProjectID}
}
issueData := &jira.Issue{
Fields: fields,
@@ -0,0 +1,312 @@
// Package jsonutil provides a function for decoding JSON
// into a GraphQL query data structure.
//
// Taken from: https://github.com/shurcooL/graphql/blob/ed46e5a4646634fc16cb07c3b8db389542cc8847/internal/jsonutil/graphql.go
package jsonutil
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"reflect"
"strings"
)
// UnmarshalGraphQL parses the JSON-encoded GraphQL response data and stores
// the result in the GraphQL query data structure pointed to by v.
//
// The implementation is created on top of the JSON tokenizer available
// in "encoding/json".Decoder.
func UnmarshalGraphQL(data []byte, v any) error {
dec := json.NewDecoder(bytes.NewReader(data))
dec.UseNumber()
err := (&decoder{tokenizer: dec}).Decode(v)
if err != nil {
return err
}
tok, err := dec.Token()
switch err {
case io.EOF:
// Expect to get io.EOF. There shouldn't be any more
// tokens left after we've decoded v successfully.
return nil
case nil:
return fmt.Errorf("invalid token '%v' after top-level value", tok)
default:
return err
}
}
// decoder is a JSON decoder that performs custom unmarshaling behavior
// for GraphQL query data structures. It's implemented on top of a JSON tokenizer.
type decoder struct {
tokenizer interface {
Token() (json.Token, error)
}
// Stack of what part of input JSON we're in the middle of - objects, arrays.
parseState []json.Delim
// Stacks of values where to unmarshal.
// The top of each stack is the reflect.Value where to unmarshal next JSON value.
//
// The reason there's more than one stack is because we might be unmarshaling
// a single JSON value into multiple GraphQL fragments or embedded structs, so
// we keep track of them all.
vs [][]reflect.Value
}
// Decode decodes a single JSON value from d.tokenizer into v.
func (d *decoder) Decode(v any) error {
rv := reflect.ValueOf(v)
if rv.Kind() != reflect.Ptr {
return fmt.Errorf("cannot decode into non-pointer %T", v)
}
d.vs = [][]reflect.Value{{rv.Elem()}}
return d.decode()
}
// decode decodes a single JSON value from d.tokenizer into d.vs.
func (d *decoder) decode() error {
// The loop invariant is that the top of each d.vs stack
// is where we try to unmarshal the next JSON value we see.
for len(d.vs) > 0 {
tok, err := d.tokenizer.Token()
if err == io.EOF {
return errors.New("unexpected end of JSON input")
} else if err != nil {
return err
}
switch {
// Are we inside an object and seeing next key (rather than end of object)?
case d.state() == '{' && tok != json.Delim('}'):
key, ok := tok.(string)
if !ok {
return errors.New("unexpected non-key in JSON input")
}
someFieldExist := false
for i := range d.vs {
v := d.vs[i][len(d.vs[i])-1]
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
var f reflect.Value
if v.Kind() == reflect.Struct {
f = fieldByGraphQLName(v, key)
if f.IsValid() {
someFieldExist = true
}
}
d.vs[i] = append(d.vs[i], f)
}
if !someFieldExist {
return fmt.Errorf("struct field for %q doesn't exist in any of %v places to unmarshal", key, len(d.vs))
}
// We've just consumed the current token, which was the key.
// Read the next token, which should be the value, and let the rest of code process it.
tok, err = d.tokenizer.Token()
if err == io.EOF {
return errors.New("unexpected end of JSON input")
} else if err != nil {
return err
}
// Are we inside an array and seeing next value (rather than end of array)?
case d.state() == '[' && tok != json.Delim(']'):
someSliceExist := false
for i := range d.vs {
v := d.vs[i][len(d.vs[i])-1]
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
var f reflect.Value
if v.Kind() == reflect.Slice {
v.Set(reflect.Append(v, reflect.Zero(v.Type().Elem()))) // v = append(v, T).
f = v.Index(v.Len() - 1)
someSliceExist = true
}
d.vs[i] = append(d.vs[i], f)
}
if !someSliceExist {
return fmt.Errorf("slice doesn't exist in any of %v places to unmarshal", len(d.vs))
}
}
switch tok := tok.(type) {
case string, json.Number, bool, nil:
// Value.
for i := range d.vs {
v := d.vs[i][len(d.vs[i])-1]
if !v.IsValid() {
continue
}
err := unmarshalValue(tok, v)
if err != nil {
return err
}
}
d.popAllVs()
case json.Delim:
switch tok {
case '{':
// Start of object.
d.pushState(tok)
frontier := make([]reflect.Value, len(d.vs)) // Places to look for GraphQL fragments/embedded structs.
for i := range d.vs {
v := d.vs[i][len(d.vs[i])-1]
frontier[i] = v
// TODO: Do this recursively or not? Add a test case if needed.
if v.Kind() == reflect.Ptr && v.IsNil() {
v.Set(reflect.New(v.Type().Elem())) // v = new(T).
}
}
// Find GraphQL fragments/embedded structs recursively, adding to frontier
// as new ones are discovered and exploring them further.
for len(frontier) > 0 {
v := frontier[0]
frontier = frontier[1:]
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Struct {
continue
}
for i := 0; i < v.NumField(); i++ {
if isGraphQLFragment(v.Type().Field(i)) || v.Type().Field(i).Anonymous {
// Add GraphQL fragment or embedded struct.
d.vs = append(d.vs, []reflect.Value{v.Field(i)})
frontier = append(frontier, v.Field(i))
}
}
}
case '[':
// Start of array.
d.pushState(tok)
for i := range d.vs {
v := d.vs[i][len(d.vs[i])-1]
// TODO: Confirm this is needed, write a test case.
//if v.Kind() == reflect.Ptr && v.IsNil() {
// v.Set(reflect.New(v.Type().Elem())) // v = new(T).
//}
// Reset slice to empty (in case it had non-zero initial value).
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Slice {
continue
}
v.Set(reflect.MakeSlice(v.Type(), 0, 0)) // v = make(T, 0, 0).
}
case '}', ']':
// End of object or array.
d.popAllVs()
d.popState()
default:
return errors.New("unexpected delimiter in JSON input")
}
default:
return errors.New("unexpected token in JSON input")
}
}
return nil
}
// pushState pushes a new parse state s onto the stack.
func (d *decoder) pushState(s json.Delim) {
d.parseState = append(d.parseState, s)
}
// popState pops a parse state (already obtained) off the stack.
// The stack must be non-empty.
func (d *decoder) popState() {
d.parseState = d.parseState[:len(d.parseState)-1]
}
// state reports the parse state on top of stack, or 0 if empty.
func (d *decoder) state() json.Delim {
if len(d.parseState) == 0 {
return 0
}
return d.parseState[len(d.parseState)-1]
}
// popAllVs pops from all d.vs stacks, keeping only non-empty ones.
func (d *decoder) popAllVs() {
var nonEmpty [][]reflect.Value
for i := range d.vs {
d.vs[i] = d.vs[i][:len(d.vs[i])-1]
if len(d.vs[i]) > 0 {
nonEmpty = append(nonEmpty, d.vs[i])
}
}
d.vs = nonEmpty
}
// fieldByGraphQLName returns an exported struct field of struct v
// that matches GraphQL name, or invalid reflect.Value if none found.
func fieldByGraphQLName(v reflect.Value, name string) reflect.Value {
for i := 0; i < v.NumField(); i++ {
if v.Type().Field(i).PkgPath != "" {
// Skip unexported field.
continue
}
if hasGraphQLName(v.Type().Field(i), name) {
return v.Field(i)
}
}
return reflect.Value{}
}
// hasGraphQLName reports whether struct field f has GraphQL name.
func hasGraphQLName(f reflect.StructField, name string) bool {
value, ok := f.Tag.Lookup("graphql")
if !ok {
// TODO: caseconv package is relatively slow. Optimize it, then consider using it here.
//return caseconv.MixedCapsToLowerCamelCase(f.Name) == name
return strings.EqualFold(f.Name, name)
}
value = strings.TrimSpace(value) // TODO: Parse better.
if strings.HasPrefix(value, "...") {
// GraphQL fragment. It doesn't have a name.
return false
}
// Cut off anything that follows the field name,
// such as field arguments, aliases, directives.
if i := strings.IndexAny(value, "(:@"); i != -1 {
value = value[:i]
}
return strings.TrimSpace(value) == name
}
// isGraphQLFragment reports whether struct field f is a GraphQL fragment.
func isGraphQLFragment(f reflect.StructField) bool {
value, ok := f.Tag.Lookup("graphql")
if !ok {
return false
}
value = strings.TrimSpace(value) // TODO: Parse better.
return strings.HasPrefix(value, "...")
}
// unmarshalValue unmarshals JSON value into v.
// v must be addressable and not obtained by the use of unexported
// struct fields, otherwise unmarshalValue will panic.
func unmarshalValue(value json.Token, v reflect.Value) error {
b, err := json.Marshal(value) // TODO: Short-circuit (if profiling says it's worth it).
if err != nil {
return err
}
return json.Unmarshal(b, v.Addr().Interface())
}
+409
View File
@@ -0,0 +1,409 @@
package linear
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"github.com/shurcooL/graphql"
"github.com/projectdiscovery/nuclei/v3/pkg/model/types/severity"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/exporters/markdown/util"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/format"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/filters"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/trackers/linear/jsonutil"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/retryablehttp-go"
)
// Integration is a client for linear issue tracker integration
type Integration struct {
url string
httpclient *http.Client
options *Options
}
// Options contains the configuration options for linear issue tracker client
type Options struct {
// APIKey is the API key for linear account.
APIKey string `yaml:"api-key" validate:"required"`
// AllowList contains a list of allowed events for this tracker
AllowList *filters.Filter `yaml:"allow-list"`
// DenyList contains a list of denied events for this tracker
DenyList *filters.Filter `yaml:"deny-list"`
// TeamID is the team id for the project
TeamID string `yaml:"team-id"`
// ProjectID is the project id for the project
ProjectID string `yaml:"project-id"`
// DuplicateIssueCheck is a bool to enable duplicate tracking issue check and update the newest
DuplicateIssueCheck bool `yaml:"duplicate-issue-check" default:"false"`
// OpenStateID is the id of the open state for the project
OpenStateID string `yaml:"open-state-id"`
HttpClient *retryablehttp.Client `yaml:"-"`
OmitRaw bool `yaml:"-"`
}
// New creates a new issue tracker integration client based on options.
func New(options *Options) (*Integration, error) {
httpClient := &http.Client{
Transport: &addHeaderTransport{
T: http.DefaultTransport,
Key: options.APIKey,
},
}
integration := &Integration{
url: "https://api.linear.app/graphql",
options: options,
httpclient: httpClient,
}
return integration, nil
}
// CreateIssue creates an issue in the tracker
func (i *Integration) CreateIssue(event *output.ResultEvent) (*filters.CreateIssueResponse, error) {
summary := format.Summary(event)
description := format.CreateReportDescription(event, util.MarkdownFormatter{}, i.options.OmitRaw)
_ = description
ctx := context.Background()
var err error
var existingIssue *linearIssue
if i.options.DuplicateIssueCheck {
existingIssue, err = i.findIssueByTitle(ctx, summary)
if err != nil && !errors.Is(err, io.EOF) {
return nil, err
}
}
if existingIssue == nil {
// Create a new issue
createdIssue, err := i.createIssueLinear(ctx, summary, description, priorityFromSeverity(event.Info.SeverityHolder.Severity))
if err != nil {
return nil, err
}
return &filters.CreateIssueResponse{
IssueID: types.ToString(createdIssue.ID),
IssueURL: types.ToString(createdIssue.URL),
}, nil
} else {
if existingIssue.State.Name == "Done" {
// Update the issue state to open
var issueUpdateInput struct {
StateID string `json:"stateId"`
}
issueUpdateInput.StateID = i.options.OpenStateID
variables := map[string]interface{}{
"issueUpdateInput": issueUpdateInput,
"issueID": types.ToString(existingIssue.ID),
}
var resp struct {
IssueUpdate struct {
LastSyncID int `json:"lastSyncId"`
}
}
err := i.doGraphqlRequest(ctx, existingIssueUpdateStateMutation, &resp, variables, "IssueUpdate")
if err != nil {
return nil, fmt.Errorf("error reopening issue %s: %s", existingIssue.ID, err)
}
}
commentInput := map[string]interface{}{
"issueId": types.ToString(existingIssue.ID),
"body": description,
}
variables := map[string]interface{}{
"commentCreateInput": commentInput,
}
var resp struct {
CommentCreate struct {
LastSyncID int `json:"lastSyncId"`
}
}
err := i.doGraphqlRequest(ctx, commentCreateExistingTicketMutation, &resp, variables, "CommentCreate")
if err != nil {
return nil, fmt.Errorf("error commenting on issue %s: %s", existingIssue.ID, err)
}
return &filters.CreateIssueResponse{
IssueID: types.ToString(existingIssue.ID),
IssueURL: types.ToString(existingIssue.URL),
}, nil
}
}
func priorityFromSeverity(sev severity.Severity) float64 {
switch sev {
case severity.Critical:
return linearPriorityCritical
case severity.High:
return linearPriorityHigh
case severity.Medium:
return linearPriorityMedium
case severity.Low:
return linearPriorityLow
default:
return linearPriorityNone
}
}
type createIssueMutation struct {
IssueCreate struct {
Issue struct {
ID graphql.ID
Title graphql.String
Identifier graphql.String
State struct {
Name graphql.String
}
URL graphql.String
}
}
}
const (
createIssueGraphQLMutation = `mutation CreateIssue($input: IssueCreateInput!) {
issueCreate(input: $input) {
issue {
id
title
identifier
state {
name
}
url
}
}
}`
searchExistingTicketQuery = `query ($teamID: ID, $projectID: ID, $title: String!) {
issues(filter: {
title: { eq: $title },
team: { id: { eq: $teamID } }
project: { id: { eq: $projectID } }
}) {
nodes {
id
title
identifier
state {
name
}
url
}
}
}
`
existingIssueUpdateStateMutation = `mutation IssueUpdate($issueUpdateInput: IssueUpdateInput!, $issueID: String!) {
issueUpdate(input: $issueUpdateInput, id: $issueID) {
lastSyncId
}
}
`
commentCreateExistingTicketMutation = `mutation CommentCreate($commentCreateInput: CommentCreateInput!) {
commentCreate(input: $commentCreateInput) {
lastSyncId
}
}
`
)
func (i *Integration) createIssueLinear(ctx context.Context, title, description string, priority float64) (*linearIssue, error) {
var mutation createIssueMutation
input := map[string]interface{}{
"title": title,
"description": description,
"priority": priority,
}
if i.options.TeamID != "" {
input["teamId"] = graphql.ID(i.options.TeamID)
}
if i.options.ProjectID != "" {
input["projectId"] = i.options.ProjectID
}
variables := map[string]interface{}{
"input": input,
}
err := i.doGraphqlRequest(ctx, createIssueGraphQLMutation, &mutation, variables, "CreateIssue")
if err != nil {
return nil, err
}
return &linearIssue{
ID: mutation.IssueCreate.Issue.ID,
Title: mutation.IssueCreate.Issue.Title,
Identifier: mutation.IssueCreate.Issue.Identifier,
State: struct {
Name graphql.String
}{
Name: mutation.IssueCreate.Issue.State.Name,
},
URL: mutation.IssueCreate.Issue.URL,
}, nil
}
func (i *Integration) findIssueByTitle(ctx context.Context, title string) (*linearIssue, error) {
var query findExistingIssuesSearch
variables := map[string]interface{}{
"title": graphql.String(title),
}
if i.options.TeamID != "" {
variables["teamId"] = graphql.ID(i.options.TeamID)
}
if i.options.ProjectID != "" {
variables["projectID"] = graphql.ID(i.options.ProjectID)
}
err := i.doGraphqlRequest(ctx, searchExistingTicketQuery, &query, variables, "")
if err != nil {
return nil, err
}
if len(query.Issues.Nodes) > 0 {
return &query.Issues.Nodes[0], nil
}
return nil, io.EOF
}
func (i *Integration) Name() string {
return "linear"
}
func (i *Integration) CloseIssue(event *output.ResultEvent) error {
// TODO: Unimplemented for now as not used in many places
// and overhead of maintaining our own API for this.
// This is too much code as it is :(
return nil
}
// ShouldFilter determines if an issue should be logged to this tracker
func (i *Integration) ShouldFilter(event *output.ResultEvent) bool {
if i.options.AllowList != nil && !i.options.AllowList.GetMatch(event) {
return false
}
if i.options.DenyList != nil && i.options.DenyList.GetMatch(event) {
return false
}
return true
}
type linearIssue struct {
ID graphql.ID
Title graphql.String
Identifier graphql.String
State struct {
Name graphql.String
}
URL graphql.String
}
type findExistingIssuesSearch struct {
Issues struct {
Nodes []linearIssue
}
}
// Custom transport to add the API key to the header
type addHeaderTransport struct {
T http.RoundTripper
Key string
}
func (adt *addHeaderTransport) RoundTrip(req *http.Request) (*http.Response, error) {
req.Header.Add("Authorization", adt.Key)
return adt.T.RoundTrip(req)
}
const (
linearPriorityNone = float64(0)
linearPriorityCritical = float64(1)
linearPriorityHigh = float64(2)
linearPriorityMedium = float64(3)
linearPriorityLow = float64(4)
)
// errors represents the "errors" array in a response from a GraphQL server.
// If returned via error interface, the slice is expected to contain at least 1 element.
//
// Specification: https://spec.graphql.org/October2021/#sec-Errors.
type errorsGraphql []struct {
Message string
Locations []struct {
Line int
Column int
}
}
// Error implements error interface.
func (e errorsGraphql) Error() string {
return e[0].Message
}
// do executes a single GraphQL operation.
func (i *Integration) doGraphqlRequest(ctx context.Context, query string, v any, variables map[string]any, operationName string) error {
in := struct {
Query string `json:"query"`
Variables map[string]any `json:"variables,omitempty"`
OperationName string `json:"operationName,omitempty"`
}{
Query: query,
Variables: variables,
OperationName: operationName,
}
var buf bytes.Buffer
err := json.NewEncoder(&buf).Encode(in)
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, i.url, &buf)
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
resp, err := i.httpclient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf("non-200 OK status code: %v body: %q", resp.Status, body)
}
var out struct {
Data *json.RawMessage
Errors errorsGraphql
//Extensions any // Unused.
}
err = json.NewDecoder(resp.Body).Decode(&out)
if err != nil {
return err
}
if out.Data != nil {
err := jsonutil.UnmarshalGraphQL(*out.Data, v)
if err != nil {
return err
}
}
if len(out.Errors) > 0 {
return out.Errors
}
return nil
}
+5 -1
View File
@@ -52,6 +52,10 @@ func (s *ScanContext) Context() context.Context {
return s.ctx
}
func (s *ScanContext) GenerateErrorMessage() string {
return joinErrors(s.errors)
}
// GenerateResult returns final results slice from all events
func (s *ScanContext) GenerateResult() []*output.ResultEvent {
s.m.Lock()
@@ -96,7 +100,7 @@ func (s *ScanContext) LogError(err error) {
}
s.errors = append(s.errors, err)
errorMessage := joinErrors(s.errors)
errorMessage := s.GenerateErrorMessage()
for _, result := range s.results {
result.Error = errorMessage
+33 -24
View File
@@ -11,7 +11,6 @@ import (
"errors"
"fmt"
"os"
"regexp"
"strings"
"sync"
@@ -21,18 +20,21 @@ import (
)
var (
ReDigest = regexp.MustCompile(`(?m)^#\sdigest:\s.+$`)
ErrUnknownAlgorithm = errors.New("unknown algorithm")
SignaturePattern = "# digest: "
SignatureFmt = SignaturePattern + "%x" + ":%v" // `#digest: <signature>:<fragment>`
)
func RemoveSignatureFromData(data []byte) []byte {
return bytes.Trim(ReDigest.ReplaceAll(data, []byte("")), "\n")
}
func GetSignatureFromData(data []byte) []byte {
return ReDigest.Find(data)
// ExtractSignatureAndContent extracts the signature (if present) and returns the content without the signature
func ExtractSignatureAndContent(data []byte) (signature, content []byte) {
dataStr := string(data)
if idx := strings.LastIndex(dataStr, SignaturePattern); idx != -1 {
signature = []byte(strings.TrimSpace(dataStr[idx:]))
content = []byte(strings.TrimSpace(dataStr[:idx]))
} else {
content = data
}
return
}
// SignableTemplate is a template that can be signed
@@ -69,26 +71,29 @@ func (t *TemplateSigner) GetUserFragment() string {
// Sign signs the given template with the template signer and returns the signature
func (t *TemplateSigner) Sign(data []byte, tmpl SignableTemplate) (string, error) {
existingSignature, content := ExtractSignatureAndContent(data)
// while re-signing template check if it has a code protocol
// if it does then verify that it is signed by current signer
// if not then return error
if tmpl.HasCodeProtocol() {
sig := GetSignatureFromData(data)
arr := strings.SplitN(string(sig), ":", 3)
if len(arr) == 2 {
// signature has no fragment
return "", errorutil.NewWithTag("signer", "re-signing code templates are not allowed for security reasons.")
}
if len(arr) == 3 {
// signature has fragment verify if it is equal to current fragment
fragment := t.GetUserFragment()
if fragment != arr[2] {
if len(existingSignature) > 0 {
arr := strings.SplitN(string(existingSignature), ":", 3)
if len(arr) == 2 {
// signature has no fragment
return "", errorutil.NewWithTag("signer", "re-signing code templates are not allowed for security reasons.")
}
if len(arr) == 3 {
// signature has fragment verify if it is equal to current fragment
fragment := t.GetUserFragment()
if fragment != arr[2] {
return "", errorutil.NewWithTag("signer", "re-signing code templates are not allowed for security reasons.")
}
}
}
}
buff := bytes.NewBuffer(RemoveSignatureFromData(data))
buff := bytes.NewBuffer(content)
// if file has any imports process them
for _, file := range tmpl.GetFileImports() {
bin, err := os.ReadFile(file)
@@ -123,12 +128,16 @@ func (t *TemplateSigner) sign(data []byte) (string, error) {
// Verify verifies the given template with the template signer
func (t *TemplateSigner) Verify(data []byte, tmpl SignableTemplate) (bool, error) {
digestData := ReDigest.Find(data)
if len(digestData) == 0 {
return false, errors.New("digest not found")
signature, content := ExtractSignatureAndContent(data)
if len(signature) == 0 {
return false, errors.New("no signature found")
}
digestData = bytes.TrimSpace(bytes.TrimPrefix(digestData, []byte(SignaturePattern)))
if !bytes.HasPrefix(signature, []byte(SignaturePattern)) {
return false, errors.New("signature must be at the end of the template")
}
digestData := bytes.TrimSpace(bytes.TrimPrefix(signature, []byte(SignaturePattern)))
// remove fragment from digest as it is used for re-signing purposes only
digestString := strings.TrimSuffix(string(digestData), ":"+t.GetUserFragment())
digest, err := hex.DecodeString(digestString)
@@ -136,7 +145,7 @@ func (t *TemplateSigner) Verify(data []byte, tmpl SignableTemplate) (bool, error
return false, err
}
buff := bytes.NewBuffer(RemoveSignatureFromData(data))
buff := bytes.NewBuffer(content)
// if file has any imports process them
for _, file := range tmpl.GetFileImports() {
bin, err := os.ReadFile(file)
+126
View File
@@ -0,0 +1,126 @@
package signer
import (
"bytes"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
testCertFile = "../../../integration_tests/protocols/keys/ci.crt"
testKeyFile = "../../../integration_tests/protocols/keys/ci-private-key.pem"
)
type mockSignableTemplate struct {
imports []string
hasCode bool
}
func (m *mockSignableTemplate) GetFileImports() []string {
return m.imports
}
func (m *mockSignableTemplate) HasCodeProtocol() bool {
return m.hasCode
}
var signer, _ = NewTemplateSignerFromFiles(testCertFile, testKeyFile)
func TestTemplateSignerSignAndVerify(t *testing.T) {
tempDir := t.TempDir()
tests := []struct {
name string
data []byte
tmpl SignableTemplate
wantSignErr bool
wantVerifyErr bool
wantVerified bool
modifyAfterSign func([]byte) []byte
}{
{
name: "Simple template",
data: []byte("id: test-template\ninfo:\n name: Test Template"),
tmpl: &mockSignableTemplate{},
wantVerified: true,
},
{
name: "Template with imports",
data: []byte("id: test-template\ninfo:\n name: Test Template"),
tmpl: &mockSignableTemplate{imports: []string{
filepath.Join(tempDir, "import1.yaml"),
filepath.Join(tempDir, "import2.yaml"),
}},
wantVerified: true,
},
{
name: "Template with code protocol",
data: []byte("id: test-template\ninfo:\n name: Test Template\n\ncode:\n - engine: bash\n source: echo 'Hello, World!'"),
tmpl: &mockSignableTemplate{hasCode: true},
wantSignErr: false,
wantVerified: true,
},
{
name: "Tampered template",
data: []byte("id: test-template\ninfo:\n name: Test Template"),
tmpl: &mockSignableTemplate{},
modifyAfterSign: func(data []byte) []byte {
signatureIndex := bytes.LastIndex(data, []byte(SignaturePattern))
if signatureIndex == -1 {
return data
}
return append(data[:signatureIndex], append([]byte("# Tampered content\n"), data[signatureIndex:]...)...)
},
wantVerified: false,
},
{
name: "Invalid signature",
data: []byte("id: test-template\ninfo:\n name: Test Template"),
tmpl: &mockSignableTemplate{},
modifyAfterSign: func(data []byte) []byte {
return append(bytes.TrimSuffix(data, []byte("\n")), []byte("\n# digest: invalid_signature:fragment")...)
},
wantVerifyErr: true,
wantVerified: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Create import files if needed
for _, imp := range tt.tmpl.GetFileImports() {
err := os.WriteFile(imp, []byte("imported content"), 0644)
require.NoError(t, err, "Failed to create import file")
}
// Sign the template
signature, err := signer.Sign(tt.data, tt.tmpl)
if tt.wantSignErr {
assert.Error(t, err, "Expected an error during signing")
return
}
require.NoError(t, err, "Failed to sign template")
// Append signature to the template data
signedData := append(tt.data, []byte("\n"+signature)...)
// Apply any modifications after signing if specified
if tt.modifyAfterSign != nil {
signedData = tt.modifyAfterSign(signedData)
}
// Verify the signature
verified, err := signer.Verify(signedData, tt.tmpl)
if tt.wantVerifyErr {
assert.Error(t, err, "Expected an error during verification")
} else {
assert.NoError(t, err, "Unexpected error during verification")
}
assert.Equal(t, tt.wantVerified, verified, "Unexpected verification result")
})
}
}
+2 -1
View File
@@ -75,11 +75,12 @@ func SignTemplate(templateSigner *signer.TemplateSigner, templatePath string) er
return ErrNotATemplate
}
if !template.Verified {
_, content := signer.ExtractSignatureAndContent(bin)
signatureData, err := templateSigner.Sign(bin, template)
if err != nil {
return err
}
buff := bytes.NewBuffer(signer.RemoveSignatureFromData(bin))
buff := bytes.NewBuffer(content)
buff.WriteString("\n" + signatureData)
return os.WriteFile(templatePath, buff.Bytes(), 0644)
}
+62 -56
View File
@@ -459,7 +459,7 @@ func init() {
Value: "HTTP response headers in name:value format",
},
}
HTTPRequestDoc.Fields = make([]encoder.Doc, 35)
HTTPRequestDoc.Fields = make([]encoder.Doc, 36)
HTTPRequestDoc.Fields[0].Name = "path"
HTTPRequestDoc.Fields[0].Type = "[]string"
HTTPRequestDoc.Fields[0].Note = ""
@@ -578,91 +578,96 @@ func init() {
HTTPRequestDoc.Fields[17].Values = []string{
"AWS",
}
HTTPRequestDoc.Fields[18].Name = "cookie-reuse"
HTTPRequestDoc.Fields[18].Name = "skip-secret-file"
HTTPRequestDoc.Fields[18].Type = "bool"
HTTPRequestDoc.Fields[18].Note = ""
HTTPRequestDoc.Fields[18].Description = "CookieReuse is an optional setting that enables cookie reuse for\nall requests defined in raw section."
HTTPRequestDoc.Fields[18].Comments[encoder.LineComment] = "CookieReuse is an optional setting that enables cookie reuse for"
HTTPRequestDoc.Fields[19].Name = "disable-cookie"
HTTPRequestDoc.Fields[18].Description = "SkipSecretFile skips the authentication or authorization configured in the secret file."
HTTPRequestDoc.Fields[18].Comments[encoder.LineComment] = "SkipSecretFile skips the authentication or authorization configured in the secret file."
HTTPRequestDoc.Fields[19].Name = "cookie-reuse"
HTTPRequestDoc.Fields[19].Type = "bool"
HTTPRequestDoc.Fields[19].Note = ""
HTTPRequestDoc.Fields[19].Description = "DisableCookie is an optional setting that disables cookie reuse"
HTTPRequestDoc.Fields[19].Comments[encoder.LineComment] = "DisableCookie is an optional setting that disables cookie reuse"
HTTPRequestDoc.Fields[20].Name = "read-all"
HTTPRequestDoc.Fields[19].Description = "CookieReuse is an optional setting that enables cookie reuse for\nall requests defined in raw section."
HTTPRequestDoc.Fields[19].Comments[encoder.LineComment] = "CookieReuse is an optional setting that enables cookie reuse for"
HTTPRequestDoc.Fields[20].Name = "disable-cookie"
HTTPRequestDoc.Fields[20].Type = "bool"
HTTPRequestDoc.Fields[20].Note = ""
HTTPRequestDoc.Fields[20].Description = "Enables force reading of the entire raw unsafe request body ignoring\nany specified content length headers."
HTTPRequestDoc.Fields[20].Comments[encoder.LineComment] = "Enables force reading of the entire raw unsafe request body ignoring"
HTTPRequestDoc.Fields[21].Name = "redirects"
HTTPRequestDoc.Fields[20].Description = "DisableCookie is an optional setting that disables cookie reuse"
HTTPRequestDoc.Fields[20].Comments[encoder.LineComment] = "DisableCookie is an optional setting that disables cookie reuse"
HTTPRequestDoc.Fields[21].Name = "read-all"
HTTPRequestDoc.Fields[21].Type = "bool"
HTTPRequestDoc.Fields[21].Note = ""
HTTPRequestDoc.Fields[21].Description = "Redirects specifies whether redirects should be followed by the HTTP Client.\n\nThis can be used in conjunction with `max-redirects` to control the HTTP request redirects."
HTTPRequestDoc.Fields[21].Comments[encoder.LineComment] = "Redirects specifies whether redirects should be followed by the HTTP Client."
HTTPRequestDoc.Fields[22].Name = "host-redirects"
HTTPRequestDoc.Fields[21].Description = "Enables force reading of the entire raw unsafe request body ignoring\nany specified content length headers."
HTTPRequestDoc.Fields[21].Comments[encoder.LineComment] = "Enables force reading of the entire raw unsafe request body ignoring"
HTTPRequestDoc.Fields[22].Name = "redirects"
HTTPRequestDoc.Fields[22].Type = "bool"
HTTPRequestDoc.Fields[22].Note = ""
HTTPRequestDoc.Fields[22].Description = "Redirects specifies whether only redirects to the same host should be followed by the HTTP Client.\n\nThis can be used in conjunction with `max-redirects` to control the HTTP request redirects."
HTTPRequestDoc.Fields[22].Comments[encoder.LineComment] = "Redirects specifies whether only redirects to the same host should be followed by the HTTP Client."
HTTPRequestDoc.Fields[23].Name = "pipeline"
HTTPRequestDoc.Fields[22].Description = "Redirects specifies whether redirects should be followed by the HTTP Client.\n\nThis can be used in conjunction with `max-redirects` to control the HTTP request redirects."
HTTPRequestDoc.Fields[22].Comments[encoder.LineComment] = "Redirects specifies whether redirects should be followed by the HTTP Client."
HTTPRequestDoc.Fields[23].Name = "host-redirects"
HTTPRequestDoc.Fields[23].Type = "bool"
HTTPRequestDoc.Fields[23].Note = ""
HTTPRequestDoc.Fields[23].Description = "Pipeline defines if the attack should be performed with HTTP 1.1 Pipelining\n\nAll requests must be idempotent (GET/POST). This can be used for race conditions/billions requests."
HTTPRequestDoc.Fields[23].Comments[encoder.LineComment] = "Pipeline defines if the attack should be performed with HTTP 1.1 Pipelining"
HTTPRequestDoc.Fields[24].Name = "unsafe"
HTTPRequestDoc.Fields[23].Description = "Redirects specifies whether only redirects to the same host should be followed by the HTTP Client.\n\nThis can be used in conjunction with `max-redirects` to control the HTTP request redirects."
HTTPRequestDoc.Fields[23].Comments[encoder.LineComment] = "Redirects specifies whether only redirects to the same host should be followed by the HTTP Client."
HTTPRequestDoc.Fields[24].Name = "pipeline"
HTTPRequestDoc.Fields[24].Type = "bool"
HTTPRequestDoc.Fields[24].Note = ""
HTTPRequestDoc.Fields[24].Description = "Unsafe specifies whether to use rawhttp engine for sending Non RFC-Compliant requests.\n\nThis uses the [rawhttp](https://github.com/projectdiscovery/rawhttp) engine to achieve complete\ncontrol over the request, with no normalization performed by the client."
HTTPRequestDoc.Fields[24].Comments[encoder.LineComment] = "Unsafe specifies whether to use rawhttp engine for sending Non RFC-Compliant requests."
HTTPRequestDoc.Fields[25].Name = "race"
HTTPRequestDoc.Fields[24].Description = "Pipeline defines if the attack should be performed with HTTP 1.1 Pipelining\n\nAll requests must be idempotent (GET/POST). This can be used for race conditions/billions requests."
HTTPRequestDoc.Fields[24].Comments[encoder.LineComment] = "Pipeline defines if the attack should be performed with HTTP 1.1 Pipelining"
HTTPRequestDoc.Fields[25].Name = "unsafe"
HTTPRequestDoc.Fields[25].Type = "bool"
HTTPRequestDoc.Fields[25].Note = ""
HTTPRequestDoc.Fields[25].Description = "Race determines if all the request have to be attempted at the same time (Race Condition)\n\nThe actual number of requests that will be sent is determined by the `race_count` field."
HTTPRequestDoc.Fields[25].Comments[encoder.LineComment] = "Race determines if all the request have to be attempted at the same time (Race Condition)"
HTTPRequestDoc.Fields[26].Name = "req-condition"
HTTPRequestDoc.Fields[25].Description = "Unsafe specifies whether to use rawhttp engine for sending Non RFC-Compliant requests.\n\nThis uses the [rawhttp](https://github.com/projectdiscovery/rawhttp) engine to achieve complete\ncontrol over the request, with no normalization performed by the client."
HTTPRequestDoc.Fields[25].Comments[encoder.LineComment] = "Unsafe specifies whether to use rawhttp engine for sending Non RFC-Compliant requests."
HTTPRequestDoc.Fields[26].Name = "race"
HTTPRequestDoc.Fields[26].Type = "bool"
HTTPRequestDoc.Fields[26].Note = ""
HTTPRequestDoc.Fields[26].Description = "ReqCondition automatically assigns numbers to requests and preserves their history.\n\nThis allows matching on them later for multi-request conditions."
HTTPRequestDoc.Fields[26].Comments[encoder.LineComment] = "ReqCondition automatically assigns numbers to requests and preserves their history."
HTTPRequestDoc.Fields[27].Name = "stop-at-first-match"
HTTPRequestDoc.Fields[26].Description = "Race determines if all the request have to be attempted at the same time (Race Condition)\n\nThe actual number of requests that will be sent is determined by the `race_count` field."
HTTPRequestDoc.Fields[26].Comments[encoder.LineComment] = "Race determines if all the request have to be attempted at the same time (Race Condition)"
HTTPRequestDoc.Fields[27].Name = "req-condition"
HTTPRequestDoc.Fields[27].Type = "bool"
HTTPRequestDoc.Fields[27].Note = ""
HTTPRequestDoc.Fields[27].Description = "StopAtFirstMatch stops the execution of the requests and template as soon as a match is found."
HTTPRequestDoc.Fields[27].Comments[encoder.LineComment] = "StopAtFirstMatch stops the execution of the requests and template as soon as a match is found."
HTTPRequestDoc.Fields[28].Name = "skip-variables-check"
HTTPRequestDoc.Fields[27].Description = "ReqCondition automatically assigns numbers to requests and preserves their history.\n\nThis allows matching on them later for multi-request conditions."
HTTPRequestDoc.Fields[27].Comments[encoder.LineComment] = "ReqCondition automatically assigns numbers to requests and preserves their history."
HTTPRequestDoc.Fields[28].Name = "stop-at-first-match"
HTTPRequestDoc.Fields[28].Type = "bool"
HTTPRequestDoc.Fields[28].Note = ""
HTTPRequestDoc.Fields[28].Description = "SkipVariablesCheck skips the check for unresolved variables in request"
HTTPRequestDoc.Fields[28].Comments[encoder.LineComment] = "SkipVariablesCheck skips the check for unresolved variables in request"
HTTPRequestDoc.Fields[29].Name = "iterate-all"
HTTPRequestDoc.Fields[28].Description = "StopAtFirstMatch stops the execution of the requests and template as soon as a match is found."
HTTPRequestDoc.Fields[28].Comments[encoder.LineComment] = "StopAtFirstMatch stops the execution of the requests and template as soon as a match is found."
HTTPRequestDoc.Fields[29].Name = "skip-variables-check"
HTTPRequestDoc.Fields[29].Type = "bool"
HTTPRequestDoc.Fields[29].Note = ""
HTTPRequestDoc.Fields[29].Description = "IterateAll iterates all the values extracted from internal extractors"
HTTPRequestDoc.Fields[29].Comments[encoder.LineComment] = "IterateAll iterates all the values extracted from internal extractors"
HTTPRequestDoc.Fields[30].Name = "digest-username"
HTTPRequestDoc.Fields[30].Type = "string"
HTTPRequestDoc.Fields[29].Description = "SkipVariablesCheck skips the check for unresolved variables in request"
HTTPRequestDoc.Fields[29].Comments[encoder.LineComment] = "SkipVariablesCheck skips the check for unresolved variables in request"
HTTPRequestDoc.Fields[30].Name = "iterate-all"
HTTPRequestDoc.Fields[30].Type = "bool"
HTTPRequestDoc.Fields[30].Note = ""
HTTPRequestDoc.Fields[30].Description = "DigestAuthUsername specifies the username for digest authentication"
HTTPRequestDoc.Fields[30].Comments[encoder.LineComment] = "DigestAuthUsername specifies the username for digest authentication"
HTTPRequestDoc.Fields[31].Name = "digest-password"
HTTPRequestDoc.Fields[30].Description = "IterateAll iterates all the values extracted from internal extractors"
HTTPRequestDoc.Fields[30].Comments[encoder.LineComment] = "IterateAll iterates all the values extracted from internal extractors"
HTTPRequestDoc.Fields[31].Name = "digest-username"
HTTPRequestDoc.Fields[31].Type = "string"
HTTPRequestDoc.Fields[31].Note = ""
HTTPRequestDoc.Fields[31].Description = "DigestAuthPassword specifies the password for digest authentication"
HTTPRequestDoc.Fields[31].Comments[encoder.LineComment] = "DigestAuthPassword specifies the password for digest authentication"
HTTPRequestDoc.Fields[32].Name = "disable-path-automerge"
HTTPRequestDoc.Fields[32].Type = "bool"
HTTPRequestDoc.Fields[31].Description = "DigestAuthUsername specifies the username for digest authentication"
HTTPRequestDoc.Fields[31].Comments[encoder.LineComment] = "DigestAuthUsername specifies the username for digest authentication"
HTTPRequestDoc.Fields[32].Name = "digest-password"
HTTPRequestDoc.Fields[32].Type = "string"
HTTPRequestDoc.Fields[32].Note = ""
HTTPRequestDoc.Fields[32].Description = "DisablePathAutomerge disables merging target url path with raw request path"
HTTPRequestDoc.Fields[32].Comments[encoder.LineComment] = "DisablePathAutomerge disables merging target url path with raw request path"
HTTPRequestDoc.Fields[33].Name = "pre-condition"
HTTPRequestDoc.Fields[33].Type = "[]matchers.Matcher"
HTTPRequestDoc.Fields[32].Description = "DigestAuthPassword specifies the password for digest authentication"
HTTPRequestDoc.Fields[32].Comments[encoder.LineComment] = "DigestAuthPassword specifies the password for digest authentication"
HTTPRequestDoc.Fields[33].Name = "disable-path-automerge"
HTTPRequestDoc.Fields[33].Type = "bool"
HTTPRequestDoc.Fields[33].Note = ""
HTTPRequestDoc.Fields[33].Description = "Fuzz PreCondition is matcher-like field to check if fuzzing should be performed on this request or not"
HTTPRequestDoc.Fields[33].Comments[encoder.LineComment] = "Fuzz PreCondition is matcher-like field to check if fuzzing should be performed on this request or not"
HTTPRequestDoc.Fields[34].Name = "pre-condition-operator"
HTTPRequestDoc.Fields[34].Type = "string"
HTTPRequestDoc.Fields[33].Description = "DisablePathAutomerge disables merging target url path with raw request path"
HTTPRequestDoc.Fields[33].Comments[encoder.LineComment] = "DisablePathAutomerge disables merging target url path with raw request path"
HTTPRequestDoc.Fields[34].Name = "pre-condition"
HTTPRequestDoc.Fields[34].Type = "[]matchers.Matcher"
HTTPRequestDoc.Fields[34].Note = ""
HTTPRequestDoc.Fields[34].Description = "FuzzPreConditionOperator is the operator between multiple PreConditions for fuzzing Default is OR"
HTTPRequestDoc.Fields[34].Comments[encoder.LineComment] = "FuzzPreConditionOperator is the operator between multiple PreConditions for fuzzing Default is OR"
HTTPRequestDoc.Fields[34].Description = "Fuzz PreCondition is matcher-like field to check if fuzzing should be performed on this request or not"
HTTPRequestDoc.Fields[34].Comments[encoder.LineComment] = "Fuzz PreCondition is matcher-like field to check if fuzzing should be performed on this request or not"
HTTPRequestDoc.Fields[35].Name = "pre-condition-operator"
HTTPRequestDoc.Fields[35].Type = "string"
HTTPRequestDoc.Fields[35].Note = ""
HTTPRequestDoc.Fields[35].Description = "FuzzPreConditionOperator is the operator between multiple PreConditions for fuzzing Default is OR"
HTTPRequestDoc.Fields[35].Comments[encoder.LineComment] = "FuzzPreConditionOperator is the operator between multiple PreConditions for fuzzing Default is OR"
GENERATORSAttackTypeHolderDoc.Type = "generators.AttackTypeHolder"
GENERATORSAttackTypeHolderDoc.Comments[encoder.LineComment] = " AttackTypeHolder is used to hold internal type of the protocol"
@@ -1607,6 +1612,7 @@ func init() {
"deleteheader",
"setbody",
"waitevent",
"dialog",
"keyboard",
"debug",
"sleep",
+60 -1
View File
@@ -10,6 +10,7 @@ import (
"github.com/dop251/goja"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/js/compiler"
"github.com/projectdiscovery/nuclei/v3/pkg/operators"
"github.com/projectdiscovery/nuclei/v3/pkg/operators/common/dsl"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
@@ -19,6 +20,8 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/flow"
"github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/generic"
"github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/multiproto"
"github.com/projectdiscovery/nuclei/v3/pkg/types/nucleierr"
"github.com/projectdiscovery/utils/errkit"
)
// TemplateExecutor is an executor for a template
@@ -68,8 +71,10 @@ func (e *TemplateExecuter) Compile() error {
if cliOptions.Verbose {
rawErrorMessage := dslCompilationError.Error()
formattedErrorMessage := strings.ToUpper(rawErrorMessage[:1]) + rawErrorMessage[1:] + "."
gologger.Warning().Msgf(formattedErrorMessage)
gologger.Warning().Msg(formattedErrorMessage)
gologger.Info().Msgf("The available custom DSL functions are:")
fmt.Println(dsl.GetPrintableDslFunctionSignatures(cliOptions.NoColor))
}
}
@@ -124,6 +129,8 @@ func (e *TemplateExecuter) Execute(ctx *scan.ScanContext) (bool, error) {
executed := &atomic.Bool{}
// matched in this case means something was exported / written to output
matched := &atomic.Bool{}
// callbackCalled tracks if the callback was called or not
callbackCalled := &atomic.Bool{}
defer func() {
// it is essential to remove template context of `Scan i.e template x input pair`
// since it is of no use after scan is completed (regardless of success or failure)
@@ -141,6 +148,7 @@ func (e *TemplateExecuter) Execute(ctx *scan.ScanContext) (bool, error) {
}
ctx.OnResult = func(event *output.InternalWrappedEvent) {
callbackCalled.Store(true)
if event == nil {
// something went wrong
return
@@ -196,13 +204,64 @@ func (e *TemplateExecuter) Execute(ctx *scan.ScanContext) (bool, error) {
} else {
errx = e.engine.ExecuteWithResults(ctx)
}
ctx.LogError(errx)
if lastMatcherEvent != nil {
lastMatcherEvent.InternalEvent["error"] = tryParseCause(fmt.Errorf("%s", ctx.GenerateErrorMessage()))
writeFailureCallback(lastMatcherEvent, e.options.Options.MatcherStatus)
}
//TODO: this is a hacky way to handle the case where the callback is not called and matcher-status is true.
// This is a workaround and needs to be refactored.
// Check if callback was never called and matcher-status is true
if !callbackCalled.Load() && e.options.Options.MatcherStatus {
fakeEvent := &output.InternalWrappedEvent{
Results: []*output.ResultEvent{
{
TemplateID: e.options.TemplateID,
Info: e.options.TemplateInfo,
Type: e.getTemplateType(),
Host: ctx.Input.MetaInput.Input,
Error: tryParseCause(fmt.Errorf("%s", ctx.GenerateErrorMessage())),
},
},
OperatorsResult: &operators.Result{
Matched: false,
},
}
writeFailureCallback(fakeEvent, e.options.Options.MatcherStatus)
}
return executed.Load() || matched.Load(), errx
}
// tryParseCause tries to parse the cause of given error
// this is legacy support due to use of errorutil in existing libraries
// but this should not be required once all libraries are updated
func tryParseCause(err error) string {
errStr := ""
errX := errkit.FromError(err)
if errX != nil {
var errCause error
if len(errX.Errors()) > 1 {
errCause = errX.Errors()[0]
}
if errCause == nil {
errCause = errX
}
msg := strings.Trim(errCause.Error(), "{} ")
parts := strings.Split(msg, ":")
errCause = errkit.New("%s", parts[len(parts)-1])
errKind := errkit.GetErrorKind(err, nucleierr.ErrTemplateLogic).String()
errStr = errCause.Error()
errStr = strings.TrimSpace(strings.Replace(errStr, "errKind="+errKind, "", -1))
}
return errStr
}
// ExecuteWithResults executes the protocol requests and returns results instead of writing them.
func (e *TemplateExecuter) ExecuteWithResults(ctx *scan.ScanContext) ([]*output.ResultEvent, error) {
var errx error
+7
View File
@@ -13,6 +13,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/scan"
templateTypes "github.com/projectdiscovery/nuclei/v3/pkg/templates/types"
"github.com/kitabisa/go-ci"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
errorutil "github.com/projectdiscovery/utils/errors"
fileutil "github.com/projectdiscovery/utils/file"
@@ -201,7 +202,13 @@ func (f *FlowExecutor) ExecuteWithResults(ctx *scan.ScanContext) error {
}
}()
// TODO(dwisiswant0): remove this once we get the RCA.
defer func() {
if ci.IsCI() {
return
}
if r := recover(); r != nil {
f.ctx.LogError(fmt.Errorf("panic occurred while executing flow: %v", r))
}
+4
View File
@@ -220,6 +220,8 @@ type Options struct {
JSONExport string
// JSONLExport is the file to export JSONL output format to
JSONLExport string
// Redact redacts given keys in
Redact goflags.StringSlice
// EnableProgressBar enables progress bar
EnableProgressBar bool
// TemplateDisplay displays the template contents
@@ -382,6 +384,8 @@ type Options struct {
ScanID string
// ScanName is the name of the scan to be uploaded
ScanName string
// TeamID is the team ID to use for cloud upload
TeamID string
// JsConcurrency is the number of concurrent js routines to run
JsConcurrency int
// SecretsFile is file containing secrets for nuclei