Fixed missing preview response for JSON output when the verbose flag is applied

Feature: brute custom wordlists

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.goreleaser.yaml

@@ -11,6 +11,11 @@ builds:
       - linux
       - windows
       - darwin
+    flags:
+      - -trimpath
+    ldflags:
+      - -s -w -X main.build={{.Version}}
+
 
 archives:
   - format: tar.gz
@@ -26,17 +31,18 @@ archives:
         format: zip
 
 changelog:
+  disable: false
   sort: asc
   filters:
     exclude:
       - "^docs:"
       - "^test:"
   groups:
-    - title: "New Features"
-      regexp: '^(Add|add)(s|ed|ing)?'
+    - title: Features
+      regexp: "^(?i).*add.*$"
       order: 0
-    - title: "Bug Fixes"
-      regexp: '^(Fix|fix)(es|ed|ing)?'
+    - title: "Bug fixes"
+      regexp: "^(?i).*fix.*$"
       order: 1
-    - title: "Others"
+    - title: Others
       order: 999

README.md

@@ -1,8 +1,10 @@
 # sj (Swagger Jacker)
 
+![](img/sj-logo.png)
+
 sj is a command line tool designed to assist with auditing of exposed Swagger/OpenAPI definition files by checking the associated API endpoints for weak authentication. It also provides command templates for manual vulnerability testing.
 
-It does this by parsing the definition file for paths, parameters, and accepted methods before using the results with one of three commands:
+It does this by parsing the definition file for paths, parameters, and accepted methods before using the results with one of four commands:
 - `automate` - Crafts a series of requests and analyzes the status code of the response.
 - `prepare` - Generates a list of commands to use for manual testing.
 - `endpoints` - Generates a list of raw API routes. *Path values will not be replaced with test data*.
@@ -24,32 +26,165 @@ To install the latest version of the tool, run:
 
 ```bash
 $ go install github.com/BishopFox/sj@latest
+
+# Note: you may also need to place the path to your Go binaries within your PATH environment variable:
+$ export PATH=$PATH:~/go/bin
 ```
 
 ## Usage
 
 > Use the `automate` command to send a series of requests to each defined endpoint and analyze the status code of each response.
 
-![Automate Command](img/sj-automate.gif)
+```bash
+$ sj automate -u https://petstore.swagger.io/v2/swagger.json -q
+
+INFO[0000] Gathering API details.
+
+Title: Swagger Petstore
+Description: This is a sample server Petstore server.  You can find out more about Swagger at [http://swagger.io](http://swagger.io) or on [irc.freenode.net, #swagger](http://swagger.io/irc/).  For this sample, you can use the api key `special-key` to test the authorization filters.
+
+INFO[0000] Available authentication mechanisms:
+    - api_key
+    - petstore_auth
+
+INFO[0000] Endpoint accessible!                          Method=GET Status=200 Target="https://petstore.swagger.io/v2/pet/findByTags?tags=1"
+INFO[0000] Endpoint accessible!                          Method=POST Status=200 Target="https://petstore.swagger.io/v2/store/order"
+WARN[0000] Manual testing may be required.               Method=POST Status=500 Target="https://petstore.swagger.io/v2/user/createWithList"
+INFO[0000] Endpoint accessible!                          Method=POST Status=200 Target="https://petstore.swagger.io/v2/user"
+ERRO[0000] User not found                                Method=GET Status=404 Target="https://petstore.swagger.io/v2/user/test"
+WARN[0000] Manual testing may be required.               Method=PUT Status=415 Target="https://petstore.swagger.io/v2/user/test"
+INFO[0000] Endpoint accessible!                          Method=GET Status=200 Target="https://petstore.swagger.io/v2/user/login?password=test&username=test"
+WARN[0000] Manual testing may be required.               Method=POST Status=500 Target="https://petstore.swagger.io/v2/user/createWithArray"
+INFO[0000] Endpoint accessible!                          Method=GET Status=200 Target="https://petstore.swagger.io/v2/user/logout"
+WARN[0000] Manual testing may be required.               Method=POST Status=415 Target="https://petstore.swagger.io/v2/pet/test"
+ERRO[0000] Pet not found                                 Method=GET Status=404 Target="https://petstore.swagger.io/v2/pet/test"
+WARN[0000] Manual testing may be required.               Method=POST Status=415 Target="https://petstore.swagger.io/v2/pet/test/uploadImage"
+ERRO[0000] Order not found                               Method=GET Status=404 Target="https://petstore.swagger.io/v2/store/order/test"
+INFO[0000] Endpoint accessible!                          Method=GET Status=200 Target="https://petstore.swagger.io/v2/store/inventory"
+INFO[0000] Endpoint accessible!                          Method=POST Status=200 Target="https://petstore.swagger.io/v2/pet"
+WARN[0000] Manual testing may be required.               Method=PUT Status=415 Target="https://petstore.swagger.io/v2/pet"
+INFO[0001] Endpoint accessible!                          Method=GET Status=200 Target="https://petstore.swagger.io/v2/pet/findByStatus?status=1"
+```
 
 > Use the `prepare` command to prepare a list of commands for manual testing. Currently supports both `curl` and `sqlmap`. You will likely have to modify these slightly.
 
-![Prepare Command](img/sj-prepare.gif)
+```bash
+$ sj prepare -u https://petstore.swagger.io/v2/swagger.json -q
+
+INFO[0000] Gathering API details.
+
+Title: Swagger Petstore
+Description: This is a sample server Petstore server.  You can find out more about Swagger at [http://swagger.io](http://swagger.io) or on [irc.freenode.net, #swagger](http://swagger.io/irc/).  For this sample, you can use the api key `special-key` to test the authorization filters.
+
+INFO[0000] Available authentication mechanisms:
+    - api_key
+    - petstore_auth
+
+curl -sk -X PUT 'https://petstore.swagger.io/v2/user/test' -d '{"test":"test"}'
+curl -sk -X GET 'https://petstore.swagger.io/v2/user/test'
+curl -sk -X GET 'https://petstore.swagger.io/v2/pet/test'
+curl -sk -X POST 'https://petstore.swagger.io/v2/pet/test' -d '{"test":"test"}'
+curl -sk -X POST 'https://petstore.swagger.io/v2/store/order' -d '{"test":"test"}'
+curl -sk -X POST 'https://petstore.swagger.io/v2/pet' -d '{"test":"test"}'
+curl -sk -X PUT 'https://petstore.swagger.io/v2/pet' -d '{"test":"test"}'
+curl -sk -X GET 'https://petstore.swagger.io/v2/store/order/test'
+curl -sk -X GET 'https://petstore.swagger.io/v2/pet/findByTags?tags=1'
+curl -sk -X POST 'https://petstore.swagger.io/v2/user/createWithList' -d '{"test":"test"}'
+curl -sk -X POST 'https://petstore.swagger.io/v2/user' -d '{"test":"test"}'
+curl -sk -X GET 'https://petstore.swagger.io/v2/user/logout'
+curl -sk -X GET 'https://petstore.swagger.io/v2/user/login?password=test&username=test'
+curl -sk -X POST 'https://petstore.swagger.io/v2/pet/test/uploadImage' -d '{"test":"test"}'
+curl -sk -X GET 'https://petstore.swagger.io/v2/pet/findByStatus?status=1'
+curl -sk -X GET 'https://petstore.swagger.io/v2/store/inventory'
+curl -sk -X POST 'https://petstore.swagger.io/v2/user/createWithArray' -d '{"test":"test"}'
+```
 
 > Use the `endpoints` command to generate a list of raw endpoints from the provided definition file.
 
-![Endpoints Command](img/sj-endpoints.gif)
+```bash
+$ sj endpoints -u https://petstore.swagger.io/v2/swagger.json
+
+INFO[0000] Gathering endpoints.
+
+/v2/pet
+/v2/pet/findByStatus
+/v2/pet/findByTags
+/v2/pet/{petId}
+/v2/pet/{petId}/uploadImage
+/v2/store/inventory
+/v2/store/order
+/v2/store/order/{orderId}
+/v2/user
+/v2/user/createWithArray
+/v2/user/createWithList
+/v2/user/login
+/v2/user/logout
+/v2/user/{username}
+```
 
 > Use the `brute` command to send a series of requests in an attempt to find a definition file on the target.
 
 ```bash
-$ sj brute -u https://<TARGET>
-INFO[0000] Sending 2045 requests. This could take a while... 
-INFO[0111] Found operation definitions embedded in JavaScript file at https://<TARGET>/example.js 
+$ sj brute -u https://petstore.swagger.io/v2/swagger.json
+INFO[0000] Sending 2045 requests. This could take a while...
+Request: 343
+INFO[0015] Definition file found: https://petstore.swagger.io/v2/swagger
+{"...SNIP..."}
 ```
 
 ## Help
 
 A full list of commands can be found by using the `--help` flag:
 
-![Help Command](img/sj-help.gif)
+```bash
+$ sj --help
+The process of reviewing and testing exposed API definition files is often tedious and requires a large investment of time for a thorough review.
+
+sj (swaggerjacker) is a CLI tool that can be used to perform an initial check of API endpoints identified through exposed Swagger/OpenAPI definition files. 
+Once you determine what endpoints require authentication and which do not, you can use the "prepare" command to generate command templates for further (manual) testing.
+
+Example usage:
+
+Perform a quick check of endpoints which require authentication:
+$ sj automate -u https://petstore.swagger.io/v2/swagger.json
+
+Generate a list of commands to use for manual testing:
+$ sj prepare -u https://petstore.swagger.io/v2/swagger.json
+
+Generate a list of raw API routes for use with custom scripts:
+$ sj endpoints -u https://petstore.swagger.io/v2/swagger.json
+
+Perform a brute-force attack against the target to identify hidden definition files:
+$ sj brute -u https://petstore.swagger.io
+
+Usage:
+  sj [flags]
+  sj [command]
+
+Available Commands:
+  automate    Sends a series of automated requests to the discovered endpoints.
+  brute       Sends a series of automated requests to discover hidden API operation definitions.
+  endpoints   Prints a list of endpoints from the target.
+  help        Help about any command
+  prepare     Prepares a set of commands for manual testing of each endpoint.
+
+Flags:
+  -a, --agent string            Set the User-Agent string. (default "Swagger Jacker (github.com/BishopFox/sj)")
+  -b, --base-path string        Set the API base path if not defined in the definition file (i.e. /V2/).
+  -f, --format string           Declare the format of the definition file (json/yaml/yml/js). (default "json")
+  -H, --headers stringArray     Add custom headers, separated by a colon ("Name: Value"). Multiple flags are accepted.
+  -h, --help                    help for sj
+  -i, --insecure                Ignores server certificate validation.
+  -l, --local-file string       Loads the documentation from a local file.
+  -o, --outfile string          Output the results to a file. Only supported for the 'automate' and 'brute' commands at this time.
+  -p, --proxy string            Proxy host and port. Example: http://127.0.0.1:8080 (default "NOPROXY")
+  -q, --quiet                   Do not prompt for user input - uses default values for all requests.
+  -r, --randomize-user-agent    Randomizes the user agent string. Default is 'false'.
+  -s, --safe-word stringArray   Avoids 'dangerous word' check for the specified word(s). Multiple flags are accepted.
+  -T, --target string           Manually set a target for the requests to be made if separate from the host the documentation resides on.
+  -t, --timeout int             Set the request timeout period. (default 30)
+  -u, --url string              Loads the documentation file from a URL
+  -v, --version                 version for sj
+
+Use "sj [command] --help" for more information about a command.
+```

cmd/auth.go

@@ -13,22 +13,25 @@ import (
 var (
 	autoApplyAPIKey    string = "n"
 	autoApplyBasicAuth string = "n"
+	autoApplyBearer    string = "n"
 	basicAuthUser      string
 	basicAuthPass      string
 	basicAuth          []byte
 	basicAuthString    string
+	bearerToken        string
 )
 
 func CheckSecDefs(doc3 openapi3.T) (apiInQuery bool, apiKey string, apiKeyName string) {
 
-	if outputFormat == "json" {
+	if outputFormat == "json" && !quiet {
 		if len(doc3.Components.SecuritySchemes) != 0 {
 			log.Warnf("The following authentication mechanisms are supported. If necessary, supply these manually when using the JSON output format:\n")
 		}
 		for mechanism := range doc3.Components.SecuritySchemes {
 			log.Infof(`Type: %s, Scheme: %s`, doc3.Components.SecuritySchemes[mechanism].Value.Type, doc3.Components.SecuritySchemes[mechanism].Value.Scheme)
 		}
-	} else {
+		log.Warnln("Note: To avoid these messages, pass the quiet (-q) flag.")
+	} else if !quiet {
 		if len(doc3.Components.SecuritySchemes) > 0 {
 			log.Info("Available authentication mechanisms: ")
 		}
@@ -44,7 +47,7 @@ func CheckSecDefs(doc3 openapi3.T) (apiInQuery bool, apiKey string, apiKeyName s
 				fmt.Printf("    - %s\n", mechanism)
 			}
 
-			if doc3.Components.SecuritySchemes[mechanism].Value.Type == "http" && !quiet {
+			if doc3.Components.SecuritySchemes[mechanism].Value.Type == "http" {
 				if doc3.Components.SecuritySchemes[mechanism].Value.Scheme == "basic" {
 					log.Infof("Basic Authentication is accepted. Supply a username and password? (y/N)")
 					fmt.Scanln(&autoApplyBasicAuth)
@@ -64,7 +67,7 @@ func CheckSecDefs(doc3 openapi3.T) (apiInQuery bool, apiKey string, apiKeyName s
 				} else if strings.ToLower(doc3.Components.SecuritySchemes[mechanism].Value.Scheme) == "bearer" {
 					log.Warn("A bearer token is accepted. Review the spec and craft a token manually using the -H flag.")
 				}
-			} else if doc3.Components.SecuritySchemes[mechanism].Value.Type == "apiKey" && doc3.Components.SecuritySchemes[mechanism].Value.In == "query" && !quiet {
+			} else if doc3.Components.SecuritySchemes[mechanism].Value.Type == "apiKey" && doc3.Components.SecuritySchemes[mechanism].Value.In == "query" {
 				apiInQuery = true
 				log.Infof("An API key can be provided via a parameter string. Would you like to apply one? (y/N)")
 				fmt.Scanln(&autoApplyAPIKey)
@@ -75,15 +78,28 @@ func CheckSecDefs(doc3 openapi3.T) (apiInQuery bool, apiKey string, apiKeyName s
 					fmt.Scanln(&apiKey)
 					log.Infof("Using %s=%s as the API key in all requests.", apiKeyName, apiKey)
 				}
-			} else if doc3.Components.SecuritySchemes[mechanism].Value.Type == "apiKey" && doc3.Components.SecuritySchemes[mechanism].Value.In == "header" && !quiet {
-				log.Infof("An API key can be provided via the header %s. Would you like to apply one? (y/N)", doc3.Components.SecuritySchemes[mechanism].Value.Name)
-				fmt.Scanln(&autoApplyAPIKey)
-				autoApplyAPIKey = strings.ToLower(autoApplyAPIKey)
-				if autoApplyAPIKey == "y" {
-					apiKeyName = doc3.Components.SecuritySchemes[mechanism].Value.Name
-					fmt.Printf("What value would you like to use for the API key (%s)?", apiKeyName)
-					fmt.Scanln(&apiKey)
-					Headers = append(Headers, doc3.Components.SecuritySchemes[mechanism].Value.Name+": "+apiKey)
+			} else if doc3.Components.SecuritySchemes[mechanism].Value.Type == "apiKey" && doc3.Components.SecuritySchemes[mechanism].Value.In == "header" {
+				if mechanism == "bearer" {
+					log.Infof("A bearer token is accepted. Would you like to provide one? (y/N)")
+					fmt.Scanln(&autoApplyBearer)
+					autoApplyBearer = strings.ToLower(autoApplyBearer)
+					if autoApplyBearer == "y" {
+						fmt.Printf("What value would you like to use for the Bearer Token? ")
+						fmt.Scanln(&bearerToken)
+						Headers = append(Headers, "Authorization: Bearer "+bearerToken)
+					} else {
+						log.Warn("A bearer token is accepted. Review the spec and craft a header manually using the -H flag.")
+					}
+				} else {
+					log.Infof("An API key can be provided via the header %s. Would you like to apply one? (y/N)", doc3.Components.SecuritySchemes[mechanism].Value.Name)
+					fmt.Scanln(&autoApplyAPIKey)
+					autoApplyAPIKey = strings.ToLower(autoApplyAPIKey)
+					if autoApplyAPIKey == "y" {
+						apiKeyName = doc3.Components.SecuritySchemes[mechanism].Value.Name
+						fmt.Printf("What value would you like to use for the API key (%s)?", apiKeyName)
+						fmt.Scanln(&apiKey)
+						Headers = append(Headers, doc3.Components.SecuritySchemes[mechanism].Value.Name+": "+apiKey)
+					}
 				}
 			}
 		}

cmd/automate.go

@@ -13,6 +13,9 @@ import (
 
 var getAccessibleEndpoints bool
 var outputFormat string
+var responsePreviewLength int
+var verbose bool
+
 var automateCmd = &cobra.Command{
 	Use:   "automate",
 	Short: "Sends a series of automated requests to the discovered endpoints.",
@@ -24,6 +27,8 @@ responds in an abnormal way, manual testing should be conducted (prepare manual
 		if outfile != "" && strings.ToLower(outputFormat) != "" {
 			if !strings.HasSuffix(strings.ToLower(outfile), "json") && strings.ToLower(outputFormat) != "json" {
 				log.Fatal("Only the JSON output format is supported at the moment.")
+			} else if strings.HasSuffix(strings.ToLower(outfile), "json") && strings.ToLower(outputFormat) == "console" {
+				outputFormat = "json"
 			}
 		}
 
@@ -53,4 +58,7 @@ responds in an abnormal way, manual testing should be conducted (prepare manual
 func init() {
 	automateCmd.PersistentFlags().StringVarP(&outputFormat, "output-format", "F", "console", "The output format. Only 'console' (default) and 'json' are supported at the moment.")
 	automateCmd.PersistentFlags().BoolVar(&getAccessibleEndpoints, "get-accessible-endpoints", false, "Only output the accessible endpoints (those that return a 200 status code).")
+	automateCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "enable verbose mode, which shows a preview of each response.")
+	automateCmd.PersistentFlags().IntVar(&responsePreviewLength, "response-preview-length", 50, "sets the response preview length when using verbose output.")
+
 }

cmd/brute.go

@@ -1,6 +1,7 @@
 package cmd
 
 import (
+	"bufio"
 	"encoding/json"
 	"fmt"
 	"net/http"
@@ -16,126 +17,11 @@ import (
 	log "github.com/sirupsen/logrus"
 )
 
-var (
-	prefixDirs = []string{
-		"/docs",
-		"",
-		"/swagger",
-		"/swagger/docs",
-		"/swagger/v1",
-		"/swagger/v2",
-		"/swagger/v3",
-		"/swagger/static",
-		"/swagger/ui",
-		"/swagger-ui",
-		"/api-docs",
-		"/api-docs/v1",
-		"/api-docs/v2",
-		"/apidocs",
-		"/api",
-		"/api/v1",
-		"/api/v2",
-		"/api/v3",
-		"/v1",
-		"/v2",
-		"/v3",
-		"/doc",
-		"/docs/swagger",
-		"/docs/swagger/v1",
-		"/docs/swagger/v2",
-		"/docs/swagger-ui",
-		"/docs/swagger-ui/v1",
-		"/docs/swagger-ui/v2",
-		"/docs/v1",
-		"/docs/v2",
-		"/docs/v3",
-		"/public",
-	}
-	jsonEndpoints = []string{
-		"",
-		"/index",
-		"/swagger",
-		"/swagger-ui",
-		"/swagger-resources",
-		"/swagger-config",
-		"/openapi",
-		"/api",
-		"/api-docs",
-		"/apidocs",
-		"/v1",
-		"/v2",
-		"/v3",
-		"/doc",
-		"/docs",
-		"/apispec",
-		"/apispec_1",
-		"/api-merged",
-	}
-	javascriptEndpoints = []string{
-		"/swagger-ui-init",
-		"/swagger-ui-bundle",
-		"/swagger-ui-standalone-preset",
-		"/swagger-ui",
-		"/swagger-ui.min",
-		"/swagger-ui-es-bundle-core",
-		"/swagger-ui-es-bundle",
-		"/swagger-ui-standalone-preset",
-		"/swagger-ui-layout",
-		"/swagger-ui-plugins",
-	}
-)
+var endpointWordlist string
 
-func makeURLs(target string, endpoints []string, fileExtension string) []string {
-	urls := []string{}
-	for _, dir := range prefixDirs {
-		for _, endpoint := range endpoints {
-			if dir == "" && endpoint == "" {
-				continue
-			}
-			targetURL := target + dir + endpoint + fileExtension
-			urls = append(urls, targetURL)
-
-		}
-	}
-	return urls
-}
-
-func findDefinitionFile(urls []string, client http.Client) (bool, *openapi3.T) {
-	for i, url := range urls {
-		ct := CheckContentType(client, url)
-		if strings.Contains(ct, "application/json") {
-			bodyBytes, _, _ := MakeRequest(client, "GET", url, timeout, nil)
-			if bodyBytes != nil {
-				checkSpec := UnmarshalSpec(bodyBytes)
-				if (strings.HasPrefix(checkSpec.OpenAPI, "2") || strings.HasPrefix(checkSpec.OpenAPI, "3")) && checkSpec.Paths != nil {
-					log.Infof("\nDefinition file found: %s\n", url)
-					return true, checkSpec
-				}
-			}
-		} else if strings.Contains(ct, "application/javascript") {
-			bodyBytes, bodyString, _ := MakeRequest(client, "GET", url, timeout, nil)
-			if bodyBytes != nil {
-				regexPattern := regexp.MustCompile(`(?s)let\s+(\w+)\s*=\s*({.*?});`)
-				matches := regexPattern.FindAllStringSubmatch(bodyString, -1)
-				for _, match := range matches {
-					jsonContent := match[2]
-					checkSpec := UnmarshalSpec([]byte(jsonContent))
-					if strings.HasPrefix(checkSpec.OpenAPI, "2") || strings.HasPrefix(checkSpec.OpenAPI, "3") {
-						log.Infof("\nFound operation definitions embedded in JavaScript file at %s\n", url)
-						return true, checkSpec
-					}
-				}
-			}
-		}
-		if i+1 == len(urls) {
-			fmt.Printf("\033[2K\r%s%d\n", "Request: ", i+1)
-		} else {
-			fmt.Printf("\033[2K\r%s%d", "Request: ", i+1)
-		}
-
-	}
-	return false, nil
-}
+var prefixDirs []string = []string{"", "/swagger", "/swagger/docs", "/swagger/latest", "/swagger/v1", "/swagger/v2", "/swagger/v3", "/swagger/static", "/swagger/ui", "/swagger-ui", "/api-docs", "/api-docs/v1", "/api-docs/v2", "/apidocs", "/api", "/api/v1", "/api/v2", "/api/v3", "/v1", "/v2", "/v3", "/doc", "/docs", "/docs/swagger", "/docs/swagger/v1", "/docs/swagger/v2", "/docs/swagger-ui", "/docs/swagger-ui/v1", "/docs/swagger-ui/v2", "/docs/v1", "/docs/v2", "/docs/v3", "/public", "/redoc"}
+var jsonEndpoints []string = []string{"", "/index", "/swagger", "/swagger-ui", "/swagger-resources", "/swagger-config", "/openapi", "/api", "/api-docs", "/apidocs", "/v1", "/v2", "/v3", "/doc", "/docs", "/apispec", "/apispec_1", "/api-merged"}
+var javascriptEndpoints []string = []string{"/swagger-ui-init", "/swagger-ui-bundle", "/swagger-ui-standalone-preset", "/swagger-ui", "/swagger-ui.min", "/swagger-ui-es-bundle-core", "/swagger-ui-es-bundle", "/swagger-ui-standalone-preset", "/swagger-ui-layout", "/swagger-ui-plugins"}
 
 var bruteCmd = &cobra.Command{
 	Use:   "brute",
@@ -151,11 +37,29 @@ var bruteCmd = &cobra.Command{
 			log.Warnf("Error parsing URL:%s\n", err)
 		}
 		target := u.Scheme + "://" + u.Host
+		if endpointWordlist == "" {
+			allURLs = append(allURLs, makeURLs(target, jsonEndpoints, "")...)
+			allURLs = append(allURLs, makeURLs(target, javascriptEndpoints, ".js")...)
+			allURLs = append(allURLs, makeURLs(target, jsonEndpoints, ".json")...)
+			allURLs = append(allURLs, makeURLs(target, jsonEndpoints, "/")...)
+		} else {
+			endpointList, err := os.Open(endpointWordlist)
+			if err != nil {
+				log.Fatalf("failed to open file: %s", err)
+			}
+			defer endpointList.Close()
 
-		allURLs = append(allURLs, makeURLs(target, jsonEndpoints, "")...)
-		allURLs = append(allURLs, makeURLs(target, javascriptEndpoints, ".js")...)
-		allURLs = append(allURLs, makeURLs(target, jsonEndpoints, ".json")...)
-		allURLs = append(allURLs, makeURLs(target, jsonEndpoints, "/")...)
+			scanner := bufio.NewScanner(endpointList)
+			for scanner.Scan() {
+				endpoint := scanner.Text()
+				fullURL := target + endpoint
+				allURLs = append(allURLs, fullURL)
+			}
+
+			if err := scanner.Err(); err != nil {
+				log.Fatalf("failed to read words from file: %s", err)
+			}
+		}
 		log.Infof("Sending %d requests. This could take a while...\n", len(allURLs))
 
 		specFound, definitionFile := findDefinitionFile(allURLs, client)
@@ -191,6 +95,60 @@ var bruteCmd = &cobra.Command{
 	},
 }
 
+func makeURLs(target string, endpoints []string, fileExtension string) []string {
+	urls := []string{}
+	for _, dir := range prefixDirs {
+		for _, endpoint := range endpoints {
+			if dir == "" && endpoint == "" {
+				continue
+			}
+			targetURL := target + dir + endpoint + fileExtension
+			urls = append(urls, targetURL)
+
+		}
+	}
+	return urls
+}
+
+func findDefinitionFile(urls []string, client http.Client) (bool, *openapi3.T) {
+	for i, url := range urls {
+		ct := CheckContentType(client, url)
+		if strings.Contains(ct, "application/json") {
+			bodyBytes, _, _ := MakeRequest(client, "GET", url, timeout, nil)
+			if bodyBytes != nil {
+				checkSpec := UnmarshalSpec(bodyBytes)
+				if (strings.HasPrefix(checkSpec.OpenAPI, "2") || strings.HasPrefix(checkSpec.OpenAPI, "3")) && checkSpec.Paths != nil {
+					fmt.Println("")
+					log.Infof("Definition file found: %s\n", url)
+					return true, checkSpec
+				}
+			}
+		} else if strings.Contains(ct, "application/javascript") {
+			bodyBytes, bodyString, _ := MakeRequest(client, "GET", url, timeout, nil)
+			if bodyBytes != nil {
+				regexPattern := regexp.MustCompile(`(?s)let\s+(\w+)\s*=\s*({.*?});`)
+				matches := regexPattern.FindAllStringSubmatch(bodyString, -1)
+				for _, match := range matches {
+					jsonContent := match[2]
+					checkSpec := UnmarshalSpec([]byte(jsonContent))
+					if strings.HasPrefix(checkSpec.OpenAPI, "2") || strings.HasPrefix(checkSpec.OpenAPI, "3") {
+						log.Infof("\nFound operation definitions embedded in JavaScript file at %s\n", url)
+						return true, checkSpec
+					}
+				}
+			}
+		}
+		if i == len(urls) {
+			fmt.Printf("\033[2K\r%s%d\n", "Request: ", i+1)
+		} else {
+			fmt.Printf("\033[2K\r%s%d", "Request: ", i+1)
+		}
+
+	}
+	return false, nil
+}
+
 func init() {
 	// TODO: Add a flag here (boolean) that defaults to false that will cause the program to execute 'sj automate' on the discovered definition file automatically.
+	bruteCmd.PersistentFlags().StringVarP(&endpointWordlist, "wordlist", "w", "", "The file containing a list of paths to brute force for discovery.")
 }

cmd/log.go

@@ -1,27 +1,81 @@
 package cmd
 
 import (
+	"fmt"
+	"os"
+	"strings"
+
 	log "github.com/sirupsen/logrus"
 )
 
-func writeLog(sc int, target, method string, errorMsg string) {
-	if sc != 200 {
-		if sc == 401 || sc == 403 {
-			logUnauth(sc, target, method, errorMsg)
-		} else if sc == 301 || sc == 302 {
-			logRedirect(sc, target, method)
-		} else if sc == 0 {
-			logSkipped(sc, target, method)
-		} else if sc == 404 {
-			logNotFound(sc, target, method, errorMsg)
-		} else if sc == 1 {
-			logDangerous(target, method)
+func writeLog(sc int, target, method, errorMsg, response string) {
+
+	tempResponsePreviewLength := responsePreviewLength
+
+	if len(response) < responsePreviewLength {
+		responsePreviewLength = len(response)
+	}
+
+	if outfile != "" {
+		file, err := os.OpenFile(outfile, os.O_RDWR|os.O_APPEND|os.O_CREATE, 0644)
+		if err != nil {
+			fmt.Println("Output file does not exist or cannot be created")
+			os.Exit(1)
+		}
+
+		defer file.Close()
+
+		log.SetOutput(file)
+	}
+
+	if outputFormat == "console" {
+		if strings.Contains(response, "\"") {
+			log.SetFormatter(&log.TextFormatter{DisableQuote: true, DisableTimestamp: true})
 		} else {
-			logManual(sc, target, method, errorMsg)
+			log.SetFormatter(&log.TextFormatter{DisableTimestamp: true})
 		}
 	} else {
-		logAccessible(sc, target, method)
+		log.SetFormatter(&log.JSONFormatter{DisableHTMLEscape: true, DisableTimestamp: true})
 	}
+
+	if verbose {
+		if sc != 200 {
+			if sc == 401 || sc == 403 {
+				logVerboseUnauth(sc, target, method, errorMsg, response)
+			} else if sc == 301 || sc == 302 {
+				logVerboseRedirect(sc, target, method, response)
+			} else if sc == 0 {
+				logVerboseBad(sc, target, method, response)
+			} else if sc == 404 {
+				logNotFound(sc, target, method, errorMsg)
+			} else if sc == 1 {
+				logDangerous(target, method)
+			} else {
+				logVerboseManual(sc, target, method, errorMsg, response)
+			}
+		} else {
+			logVerboseAccessible(sc, target, method, response)
+		}
+	} else {
+		if sc != 200 {
+			if sc == 401 || sc == 403 {
+				logUnauth(sc, target, method, errorMsg)
+			} else if sc == 301 || sc == 302 {
+				logRedirect(sc, target, method)
+			} else if sc == 0 {
+				logBad(sc, target, method)
+			} else if sc == 404 {
+				logNotFound(sc, target, method, errorMsg)
+			} else if sc == 1 {
+				logDangerous(target, method)
+			} else {
+				logManual(sc, target, method, errorMsg)
+			}
+		} else {
+			logAccessible(sc, target, method)
+		}
+	}
+	responsePreviewLength = tempResponsePreviewLength
 }
 
 func logAccessible(status int, target, method string) {
@@ -32,6 +86,15 @@ func logAccessible(status int, target, method string) {
 	}).Print("Endpoint accessible!")
 }
 
+func logVerboseAccessible(status int, target, method, response string) {
+	log.WithFields(log.Fields{
+		"Status":  status,
+		"Target":  target,
+		"Method":  method,
+		"Preview": response[:responsePreviewLength],
+	}).Print("Endpoint accessible!")
+}
+
 func logDangerous(target, method string) {
 	log.WithFields(log.Fields{
 		"Status": "skipped",
@@ -51,6 +114,18 @@ func logManual(status int, target, method, errorMsg string) {
 	}).Warn(errorMsg)
 }
 
+func logVerboseManual(status int, target, method, errorMsg, response string) {
+	if errorMsg == "" {
+		errorMsg = "Manual testing may be required."
+	}
+	log.WithFields(log.Fields{
+		"Status":  status,
+		"Target":  target,
+		"Method":  method,
+		"Preview": response[:responsePreviewLength],
+	}).Warn(errorMsg)
+}
+
 func logNotFound(status int, target, method, errorMsg string) {
 	if errorMsg == "" {
 		errorMsg = "Endpoint not found."
@@ -70,12 +145,30 @@ func logRedirect(status int, target, method string) {
 	}).Error("Redirect detected. This likely requires authentication.")
 }
 
-func logSkipped(status int, target, method string) {
+func logVerboseRedirect(status int, target, method, response string) {
+	log.WithFields(log.Fields{
+		"Status":  status,
+		"Target":  target,
+		"Method":  method,
+		"Preview": response[0:responsePreviewLength],
+	}).Error("Redirect detected. This likely requires authentication.")
+}
+
+func logBad(status int, target, method string) {
 	log.WithFields(log.Fields{
 		"Status": "N/A",
 		"Target": target,
 		"Method": method,
-	}).Warn("Request skipped (dangerous keyword found).")
+	}).Warn("Bad request (could not reach the target).")
+}
+
+func logVerboseBad(status int, target, method, response string) {
+	log.WithFields(log.Fields{
+		"Status":  "N/A",
+		"Target":  target,
+		"Method":  method,
+		"Preview": response[:responsePreviewLength],
+	}).Warn("Bad request (could not reach the target).")
 }
 
 func logUnauth(status int, target, method, errorMsg string) {
@@ -88,3 +181,15 @@ func logUnauth(status int, target, method, errorMsg string) {
 		"Method": method,
 	}).Error(errorMsg)
 }
+
+func logVerboseUnauth(status int, target, method, errorMsg, response string) {
+	if errorMsg == "" {
+		errorMsg = "Unauthorized."
+	}
+	log.WithFields(log.Fields{
+		"Status":  status,
+		"Target":  target,
+		"Method":  method,
+		"Preview": response[:responsePreviewLength],
+	}).Error(errorMsg)
+}

cmd/requests.go

@@ -104,7 +104,7 @@ func MakeRequest(client http.Client, method, target string, timeout int64, reqDa
 			}
 			req.Header.Set(h[0], h[1])
 		} else {
-			log.Fatal("Custom header provided cannot be used.")
+			log.Warnf("Header provided (%s) cannot be used. Headers must be in 'Key: Value' format (this may be caused by a header declared within the definition file).\n", Headers[i])
 		}
 	}
 
@@ -140,8 +140,10 @@ func MakeRequest(client http.Client, method, target string, timeout int64, reqDa
 		log.Printf("Error: %s - skipping request.", err)
 		return nil, "", 0
 	} else if err != nil && err != context.Canceled && err != io.EOF {
-		if strings.Contains(fmt.Sprint(err), "tls") && !strings.Contains(fmt.Sprint(err), "user canceled") {
+		if (strings.Contains(fmt.Sprint(err), "tls") || strings.Contains(fmt.Sprint(err), "x509")) && !strings.Contains(fmt.Sprint(err), "user canceled") {
 			log.Fatal("Try supplying the --insecure flag.")
+		} else if strings.Contains(fmt.Sprint(err), "tcp") && strings.Contains(fmt.Sprint(err), "no such host") {
+			log.Fatalf("The target '%s' is not reachable. Check the declared host(s) and supply a target manually using -T if needed.", u.Scheme+"://"+u.Host)
 		} else if strings.Contains(fmt.Sprint(err), "user canceled") {
 			return nil, "skipped", 1
 		} else {
@@ -152,16 +154,25 @@ func MakeRequest(client http.Client, method, target string, timeout int64, reqDa
 
 	bodyBytes, _ := io.ReadAll(resp.Body)
 	bodyString := string(bodyBytes)
+
+	if (resp.StatusCode == 301 || resp.StatusCode == 302) && strings.Contains(bodyString, "<html>") {
+		redirect, _ := resp.Location()
+		bodyBytes, bodyString, requestStatus = MakeRequest(client, method, redirect.Scheme+"://"+redirect.Host+redirect.Path, timeout, reqData)
+		return bodyBytes, bodyString, requestStatus
+	}
+
 	requestStatus = resp.StatusCode
 
 	return bodyBytes, bodyString, requestStatus
 }
 
-func CheckContentType(client http.Client, url string) string {
+func CheckContentType(client http.Client, target string) string {
+	u, _ := url.Parse(target)
+
 	ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeout)*time.Second)
 	defer cancel()
 
-	req, err := http.NewRequest("GET", url, nil)
+	req, err := http.NewRequest("GET", target, nil)
 	if err != nil && err != context.Canceled && err != io.EOF {
 		log.Fatal("Error: could not create HTTP request - ", err)
 	}
@@ -171,9 +182,12 @@ func CheckContentType(client http.Client, url string) string {
 		log.Printf("Error: %s - skipping request.", err)
 		return ""
 	} else if err != nil && err != context.Canceled && err != io.EOF {
-		log.Error("Error: response not received.\n", err)
-		if strings.Contains(fmt.Sprint(err), "tls") && !strings.Contains(fmt.Sprint(err), "user canceled") {
-			fmt.Println("Try supplying the --insecure flag.")
+		if (strings.Contains(fmt.Sprint(err), "tls") || strings.Contains(fmt.Sprint(err), "x509")) && !strings.Contains(fmt.Sprint(err), "user canceled") {
+			log.Fatal("Try supplying the --insecure flag.")
+		} else if strings.Contains(fmt.Sprint(err), "tcp") && strings.Contains(fmt.Sprint(err), "no such host") {
+			log.Fatalf("The target '%s' is not reachable. Check the declared host(s) and supply a target manually using -T if needed.", u.Scheme+"://"+u.Host)
+		} else {
+			log.Error("Error: response not received.\n", err)
 		}
 		return ""
 	}

cmd/root.go

@@ -46,7 +46,7 @@ $ sj brute -u https://petstore.swagger.io`,
 			log.Error("Command not specified. See the --help flag for usage.")
 		}
 	},
-	Version: "1.4.2",
+	Version: "1.7.2",
 }
 
 func Execute() {

cmd/schemas.go

@@ -2,30 +2,118 @@ package cmd
 
 import (
 	"fmt"
+	"math/rand"
+	"strings"
 
 	"github.com/getkin/kin-openapi/openapi3"
 	log "github.com/sirupsen/logrus"
 )
 
-func BuildObjectsFromSchemaDefinitions(doc openapi3.T) {
-	if doc.Components.Schemas != nil {
-		log.Infof("Building test objects from schema definitions...\n\n")
-	}
-
-	if doc.Components.Schemas != nil {
-		for definition := range doc.Components.Schemas {
-			log.Infof("%s accepts %d parameters:\n", definition, len(doc.Components.Schemas[definition].Value.Properties))
-			for property := range doc.Components.Schemas[definition].Value.Properties {
-				schemaDef := doc.Components.Schemas[definition]
-				definitionProperty := schemaDef.Value.Properties[property]
-				if definitionProperty.Value.Type == "string" {
-					log.Infof("%s=%s\n", property, "test")
-				} else if definitionProperty.Value.Type == "integer" {
-					log.Infof("%s=%d\n", property, 1)
-				} else if definitionProperty.Ref != "" {
-					fmt.Println("[!] [!] [!] " + definitionProperty.Ref) // Item $ref within a property
-				}
+type SchemaReference struct {
+	Required   []string
+	Type       string
+	Properties struct {
+		Property struct {
+			Type      string
+			Format    string
+			Example   string
+			Reference string
+			XML       struct {
+				Name      string
+				Namespace string
+				Prefix    string
+				Attribute bool
+				Wrapped   bool
 			}
 		}
 	}
 }
+
+func HandleSchemaReference(schema *openapi3.SchemaRef) {
+
+}
+
+func (s SwaggerRequest) SetParametersFromSchema(param *openapi3.ParameterRef, location, schemaRef string, req *openapi3.RequestBodyRef, counter int) SwaggerRequest {
+	if param != nil {
+		name := strings.TrimPrefix(schemaRef, "#/components/schemas/")
+		if s.Def.Components.Schemas[name] != nil {
+			schema := s.Def.Components.Schemas[name]
+			if schema.Value.Properties != nil {
+				for property := range schema.Value.Properties {
+					if schema.Value.Properties[property].Ref != "" {
+						if counter < 3 {
+							s = s.SetParametersFromSchema(param, location, schema.Value.Properties[property].Ref, req, counter+1)
+						} else {
+							log.Warnf("Nested reference encountered for %s (Property: %s). Test this endpoint manually.\n", s.URL.Scheme+"://"+s.URL.Host+s.URL.Path, property)
+							break
+						}
+					} else if location == "path" {
+						if schema.Value.Properties[property].Value.Type == "string" {
+							if strings.Contains(s.URL.Path, param.Value.Name) {
+								s.URL.Path = strings.ReplaceAll(s.URL.Path, "{"+param.Value.Name+"}", "test")
+							} else {
+								s.URL.Path = strings.ReplaceAll(s.URL.Path, "{"+strings.ToLower(param.Value.Name)+"}", "test")
+							}
+						} else {
+							if strings.Contains(s.URL.Path, param.Value.Name) {
+								s.URL.Path = strings.ReplaceAll(s.URL.Path, "{"+param.Value.Name+"}", "1")
+							} else {
+								s.URL.Path = strings.ReplaceAll(s.URL.Path, "{"+strings.ToLower(param.Value.Name)+"}", "1")
+							}
+						}
+					} else if location == "query" {
+						if schema.Value.Properties[property].Value.Type == "string" {
+							s.Query.Add(param.Value.Name, "test")
+						} else {
+							s.Query.Add(param.Value.Name, "1")
+						}
+					} else if location == "body" {
+						if schema.Value.Properties[property].Value.Type == "string" {
+							s.Body[param.Value.Name] = "test"
+						} else {
+							s.Body[param.Value.Name] = 1
+						}
+					}
+				}
+			} else if schema.Value.Enum != nil {
+				if location == "path" {
+					value := schema.Value.Enum[rand.Intn(len(schema.Value.Enum))]
+					s.URL.Path = strings.ReplaceAll(s.URL.Path, "{"+param.Value.Name+"}", fmt.Sprint(value))
+				} else if location == "query" {
+					value := schema.Value.Enum[rand.Intn(len(schema.Value.Enum))]
+					s.Query.Add(param.Value.Name, fmt.Sprint(value))
+				} else if location == "body" {
+					value := schema.Value.Enum[rand.Intn(len(schema.Value.Enum))]
+					s.Body[param.Value.Name] = value
+				}
+			}
+		}
+	} else {
+		name := strings.TrimPrefix(schemaRef, "#/components/schemas/")
+		if s.Def.Components.Schemas[name] != nil {
+			schema := s.Def.Components.Schemas[name]
+			if schema.Value.Properties != nil {
+				for property := range schema.Value.Properties {
+					if schema.Value.Properties[property].Ref != "" {
+						if counter < 3 {
+							s = s.SetParametersFromSchema(param, location, schema.Value.Properties[property].Ref, req, counter+1)
+						} else {
+							log.Warnf("Nested reference encountered for %s (Property: %s). Test this endpoint manually.\n", s.URL.Scheme+"://"+s.URL.Host+s.URL.Path, property)
+							break
+						}
+					} else {
+						if schema.Value.Properties[property].Value.Type == "string" {
+							s.Body[property] = "test"
+						} else {
+							s.Body[property] = 1
+						}
+					}
+				}
+			} else if schema.Value.Enum != nil {
+				value := schema.Value.Enum[rand.Intn(len(schema.Value.Enum))]
+				s.Body[name] = value
+			}
+		}
+	}
+	return s
+}