Merge pull request #69 from six2dez/patch-2

Update requirements.txt

README.md

@@ -7,6 +7,7 @@ Give those screenshots of yours a quick eyeballing.
 
 Eyeballer is meant for large-scope network penetration tests where you need to find "interesting" targets from a huge set of web-based hosts. Go ahead and use your favorite screenshotting tool like normal (EyeWitness or GoWitness) and then run them through Eyeballer to tell you what's likely to contain vulnerabilities, and what isn't.
 
+Give it a try live at: https://eyeballer.bishopfox.com
 
 ### Example Labels
 
@@ -59,15 +60,20 @@ For the latest pretrained weights, check out the releases here on GitHub.
 
 **Training Data** You can find our training data here:
 
-https://www.dropbox.com/s/rpylhiv2g0kokts/eyeballer-3.0.zip?dl=1
+https://www.kaggle.com/altf42600/pentest-screensots
 
 There's two things you need from the training data:
 
 1. `images/` folder, containing all the screenshots (resized down to 224x224)
 2. `labels.csv` that has all the labels
-3. `bishop-fox-pretrained-v2.h5` A pretrained weights file you can use right out of the box without training.
 
-Copy all three into the root of the Eyeballer code tree.
+Copy both into the root of the Eyeballer code tree.
+
+Additionally, you can find a pretrained weights file you can use right out of the box without training.  
+
+1. `bishop-fox-pretrained-vN.h5`
+
+It's here on GitHub, look at the `releases` section for the latest one.
 
 ## Predicting Labels
 
@@ -87,6 +93,27 @@ eyeballer.py --weights YOUR_WEIGHTS.h5 predict PATH_TO/YOUR_FILES/
 
 Eyeballer will spit the results back to you in human readable format (a `results.html` file so you can browse it easily) and machine readable format (a `results.csv` file).
 
+## Performance
+
+Eyeballer's performance is measured against an evaluation dataset, which is 20% of the overall screenshots chosen at random. Since these screenshots are never used in training, they can be an effective way to see how well the model is performing. Here are the latest results:
+
+| Overall Binary Accuracy | 93.52% |
+| ------ |:-----:|
+| **All-or-Nothing Accuracy** | **76.09%** |
+
+**Overall Binary Accuracy** is probably what you think of as the model's "accuracy". It's the chance, given any single label, that it is correct.
+
+**All-or-Nothing Accuracy** is more strict. For this, we consider all of an image's labels and consider it a failure if ANY label is wrong. This accuracy rating is the chance that the model correctly predicts all labels for any given image.
+
+| Label | Precision | Recall |
+| ------ | ------ |:-----:|
+| Custom 404 | 80.20% | 91.01% |
+| Login Page | 86.41% | 88.47% |
+| Webapp | 95.32% | 96.83% |
+| Old Looking | 91.70% | 62.20% |
+| Parked Domain | 70.99% | 66.43% |
+
+For a detailed explanation on [Precision vs Recall, check out Wikipedia](https://en.wikipedia.org/wiki/Precision_and_recall).
 
 ## Training
 To train a new model, run:

requirements.txt

@@ -4,7 +4,8 @@ matplotlib
 numpy
 pandas
 pillow
-sklearn
+scikit-learn
 tensorflow
 jinja2
 progressbar2
+scipy

utils/verify.py

@@ -15,11 +15,14 @@ with open("labels.csv", newline="") as csvfile:
     for row in csvreader:
         label_list.append(row["filename"])
 
+    print(len(file_list), "files")
+    print(len(label_list), "labels")
+
     # Loop through the file
     print("Rows in labels.csv, but no image exists:")
     for label in label_list:
         if label not in file_list:
-            print(row["filename"])
+            print(label)
 
     print("Images that exist, but don't have labels:")
     i = 0