Scale usage to many thousands of images, disables previews

- Apparently, tfjs doesn't do memory management for you!

.gitignore

@@ -104,7 +104,7 @@ venv.bak/
 .mypy_cache/
 
 #screenshot data
-*.png
+images/*.png
 *.jpg
 *.tar
 
@@ -113,4 +113,10 @@ venv.bak/
 *.model
 *.csv
 *.txt
+results.html
 !requirements.txt
+!requirements-gpu.txt
+!tests/models/*
+!tests/data/*
+!docs/*
+logs/*

README.md

@@ -0,0 +1,130 @@
+# Eyeballer
+
+![Logo](/docs/eyeballer_logo.png)
+
+
+Give those screenshots of yours a quick eyeballing.
+
+Eyeballer is meant for large-scope network penetration tests where you need to find "interesting" targets from a huge set of web-based hosts. Go ahead and use your favorite screenshotting tool like normal (EyeWitness or GoWitness) and then run them through Eyeballer to tell you what's likely to contain vulnerabilities, and what isn't.
+
+
+### Example Labels
+
+| Old-Looking Sites | Login Pages |
+| ------ |:-----:|
+| ![Sample Old-looking Page](/docs/old-looking.png) | ![Sample Login Page](/docs/login.png) |
+
+| Webapp | Custom 404's |
+| ------ |:-----:|
+| ![Sample Webapp](/docs/homepage.png) | ![Sample Custom 404](/docs/404.png) |
+
+| Parked Domains |
+| ------ |
+| ![Sample Webapp](/docs/parked.png) |
+
+## What the Labels Mean
+
+**Old-Looking Sites**
+Blocky frames, broken CSS, that certain "je ne sais quoi" of a website that looks like it was designed in the early 2000's. You know it when you see it. Old websites aren't just ugly, they're also typically super vulnerable. When you're looking to hack into something, these websites are a gold mine.
+
+**Login Pages**
+Login pages are valuable to pen testing, they indicate that there's additional functionality you don't currently have access to. It also means there's a simple follow-up process of credential enumeration attacks. You might think that you can set a simple heuristic to find login pages, but in practice it's really hard. Modern sites don't just use a simple input tag we can grep for.
+
+**Webapp**
+This tells you that there is a larger group of pages and functionality available here that can serve as surface area to attack. This is in contrast to a simple login page, with no other functionality. Or a default IIS landing page which has no other functionality. This label should indicate to you that there is a web application here to attack.
+
+**Custom 404**
+Modern sites love to have cutesy custom 404 pages with pictures of broken robots or sad looking dogs. Unfortunately, they also love to return HTTP 200 response codes while they do it. More often, the "404" page doesn't even contain the text "404" in it. These pages are typically uninteresting, despite having a lot going on visually, and Eyeballer can help you sift them out.
+
+**Parked Domains**
+Parked domains are websites that look real, but aren't valid attack surface. They're stand-in pages, usually devoid of any real functionality, consist almost entirely of ads, and are usually not run by our actual target. It's what you get when the domain specified is wrong or lapsed. Finding these pages and removing them from scope is really valuable over time.
+
+## Setup
+
+Download required packages on pip:
+```
+sudo pip3 install -r requirements.txt
+```
+
+Or if you want GPU support:
+```
+sudo pip3 install -r requirements-gpu.txt
+```
+
+**NOTE**: Setting up a GPU for use with TensorFlow is way beyond the scope of this README. There's hardware compatibility to consider, drivers to install... There's a lot. So you're just going to have to figure this part out on your own if you want a GPU. But at least from a Python package perspective, the above requirements file has you covered.
+
+**Pretrained Weights**
+
+For the latest pretrained weights, check out the releases here on GitHub.
+
+**Training Data** You can find our training data here:
+
+https://www.dropbox.com/s/rpylhiv2g0kokts/eyeballer-3.0.zip?dl=1
+
+There's two things you need from the training data:
+
+1. `images/` folder, containing all the screenshots (resized down to 224x224)
+2. `labels.csv` that has all the labels
+3. `bishop-fox-pretrained-v3.h5` A pretrained weights file you can use right out of the box without training.
+
+Copy all three into the root of the Eyeballer code tree.
+
+## Predicting Labels
+
+NOTE: For best results, make sure you screenshot your websites in a native 1.6x aspect ratio. IE: 1440x900. Eyeballer will scale the image down automatically to the right size for you, but if it's the wrong aspect ratio then it will squish in a way that will affect prediction performance.
+
+To eyeball some screenshots, just run the "predict" mode:
+
+```
+eyeballer.py --weights YOUR_WEIGHTS.h5 predict YOUR_FILE.png
+```
+
+Or for a whole directory of files:
+
+```
+eyeballer.py --weights YOUR_WEIGHTS.h5 predict PATH_TO/YOUR_FILES/
+```
+
+Eyeballer will spit the results back to you in human readable format (a `results.html` file so you can browse it easily) and machine readable format (a `results.csv` file).
+
+## Performance
+
+Eyeballer's performance is measured against an evaluation dataset, which is 20% of the overall screenshots chosen at random. Since these screenshots are never used in training, they can be an effective way to see how well the model is performing. Here are the latest results:
+
+| Overall Binary Accuracy | 93.52% |
+| ------ |:-----:|
+| **All-or-Nothing Accuracy** | **76.09%** |
+
+**Overall Binary Accuracy** is probably what you think of as the model's "accuracy". It's the chance, given any single label, that it is correct.
+
+**All-or-Nothing Accuracy** is more strict. For this, we consider all of an image's labels and consider it a failure if ANY label is wrong. This accuracy rating is the chance that the model correctly predicts all labels for any given image.
+
+| Label | Precision | Recall |
+| ------ | ------ |:-----:|
+| Custom 404 | 80.20% | 91.01% |
+| Login Page | 86.41% | 88.47% |
+| Webapp | 95.32% | 96.83% |
+| Old Looking | 91.70% | 62.20% |
+| Parked Domain | 70.99% | 66.43% |
+
+For a detailed explanation on [Precision vs Recall, check out Wikipedia](https://en.wikipedia.org/wiki/Precision_and_recall).
+
+## Training
+To train a new model, run:
+```
+eyeballer.py train
+```
+
+You'll want a machine with a good GPU for this to run in a reasonable amount of time. Setting that up is outside the scope of this readme, however.
+
+This will output a new model file (weights.h5 by default).
+
+## Evaluation
+
+You just trained a new model, cool! Let's see how well it performs against some images it's never seen before, across a variety of metrics:
+
+```
+eyeballer.py --weights YOUR_WEIGHTS.h5 evaluate
+```
+
+The output will describe the model's accuracy in both recall and precision for each of the program's labels. (Including "none of the above" as a pseudo-label)

eyeballer.py

@@ -1,8 +1,11 @@
 #!/usr/bin/env python3
+
 import click
 import csv
 
-from model import EyeballModel, DATA_LABELS
+from eyeballer.model import EyeballModel, DATA_LABELS
+from eyeballer.visualization import HeatMap
+from jinja2 import Template
 
 
 @click.group(invoke_without_command=True)
@@ -34,32 +37,81 @@ def train(ctx, graphs, batchsize, epochs):
 
 @cli.command()
 @click.argument('screenshot')
+@click.option('--heatmap', default=False, is_flag=True, help="Create a heatmap graphfor the prediction")
+@click.option('--threshold', default=.5, type=float, help="Threshold confidence for labeling")
 @click.pass_context
-def predict(ctx, screenshot):
+def predict(ctx, screenshot, heatmap, threshold):
     model = EyeballModel(**ctx.obj['model_kwargs'])
     results = model.predict(screenshot)
+
+    if heatmap:
+        # Generate a heatmap
+        HeatMap(screenshot, model, threshold).generate()
+
     if not results:
         print("Error: Input file does not exist")
     if len(results) == 1:
         print(results)
     else:
         with open("results.csv", "w", newline="") as csvfile:
-            fieldnames = ["filename", "custom404", "login", "homepage", "oldlooking"]
+            fieldnames = ["filename", "custom404", "login", "webapp", "oldlooking", "parked"]
             labelwriter = csv.DictWriter(csvfile, fieldnames=fieldnames)
             labelwriter.writeheader()
             labelwriter.writerows(results)
+
         print("Output written to results.csv")
+        buildHTML(processResults(results, threshold))
+        print("HTML written to results.html")
+
+
+def processResults(results, threshold):
+    '''Filter the initial results dictionary and reformat it for use in JS.
+
+        Keyword arguments:
+        results -- dictionary output from predict function
+
+    '''
+    jsResults = {}
+
+    for result in results:
+        positiveTags = []
+
+        for label, label_info in result.items():
+            if (label == 'filename'):
+                pass
+            elif label_info > threshold:
+                positiveTags.append(label)
+
+        jsResults[result['filename']] = positiveTags
+    return(jsResults)
+
+
+def buildHTML(jsResults):
+    '''Build HTML around the JS Dictionary that is passed from processResults.
+
+        Keyword arguments:
+        jsResults -- dictionary output from processResults function
+    '''
+    html_output = ""
+    with open("prediction_output_template.html") as template_file:
+        template = Template(template_file.read())
+        html_output = template.render(jsResults=jsResults)
+
+    with open('results.html', 'w') as file:
+        file.write(html_output)
 
 
 def pretty_print_evaluation(results):
     """Print a human-readable summary of the evaluation"""
     # We use 4.2% to handle all the way from "  0.00%" (7chars) to "100.00%" (7chars)
     for label in DATA_LABELS:
-        print(f"{label} Precision Score: {results[label]['precision']:4.2%}")
-        print(f"{label} Recall Score: {results[label]['recall']:4.2%}")
-    print(f"Overall Binary Accuracy: {results['total_binary_accuracy']:4.2%}")
-    print(f"All or nothing accuracy: {results['all_or_nothing_accuracy']:4.2%}")
-    print(f"Top 10 worst predictions: {results['top_10_worst'][1]}")
+        print("{} Precision Score: {:4.2%}".format(label, results[label]['precision']))
+        print("{} Recall Score: {:4.2%}".format(label, results[label]['recall']))
+    print("'None of the above' Precision: {:4.2%}".format(results['none_of_the_above_precision']))
+    print("'None of the above' Recall: {:4.2%}".format(results['none_of_the_above_recall']))
+    print("All or nothing Accuracy: {:4.2%}".format(results['all_or_nothing_accuracy']))
+    print("Overall Binary Accuracy: {:4.2%}".format(results['total_binary_accuracy']))
+    print("Top 10 worst predictions: {}".format(results['top_10_worst'][1]))
 
 
 @cli.command()

eyeballer/__init__.py

@@ -1 +0,0 @@
-from eyeballer.eyeballer import EyeballModel  # noqa: F401

eyeballer/augmentation.py

@@ -1,7 +1,7 @@
 import numpy as np
 
 from Augmentor.Operations import Operation
-from keras.applications.mobilenet import preprocess_input
+from tensorflow.keras.applications.mobilenet import preprocess_input
 
 
 class EyeballerAugmentation(Operation):

eyeballer/model.py

@@ -1,6 +1,7 @@
 import os
 import random
 import sys
+import progressbar
 
 # Prevent Tkinter Dependency
 import matplotlib
@@ -10,34 +11,13 @@ import matplotlib.pyplot as plt
 import numpy as np
 import pandas as pd
 import Augmentor
+import tensorflow as tf
 
-from keras import regularizers
-from keras.applications import MobileNet
-from keras.applications.mobilenet import preprocess_input
-from keras.callbacks import ModelCheckpoint
-from keras.layers import Dropout, Dense
-from keras.layers import GlobalAveragePooling2D
-from keras.models import Model
-from keras.optimizers import Adam
-from keras.preprocessing import image
-from keras.preprocessing.image import ImageDataGenerator
+from tensorflow.keras.applications.mobilenet import preprocess_input
 from sklearn.metrics import classification_report, accuracy_score, hamming_loss
-from augmentation import EyeballerAugmentation
-from keras.callbacks import Callback
+from eyeballer.augmentation import EyeballerAugmentation
 
-DATA_LABELS = ["custom404", "login", "homepage", "oldlooking"]
-
-
-class PredictionManager(Callback):
-    def __init__(self):
-        super(PredictionManager, self).__init__()
-        self.predictions = []
-
-    def getPredictions(self):
-        return np.array(self.predictions)
-
-    def on_predict_batch_end(self, batch, logs=None):
-        self.predictions.append(logs['outputs'][0].tolist()[0])
+DATA_LABELS = ["custom404", "login", "webapp", "oldlooking", "parked"]
 
 
 class EyeballModel:
@@ -47,11 +27,11 @@ class EyeballModel:
     """
     graphs_directory = "graphs/"
     checkpoint_file = "weights.h5"
-    image_dir = "images_224x224/"
+    image_dir = "images/"
     image_width, image_height = 224, 224
     input_shape = (image_width, image_height, 3)
 
-    def __init__(self, weights_file, print_summary=False, seed=None):
+    def __init__(self, weights_file, print_summary=False, seed=None, quiet=False):
         """Constructor for model class.
 
         Keyword arguments:
@@ -59,26 +39,40 @@ class EyeballModel:
         weights_file -- A filename for where to load the model's weights.
         seed -- PRNG seed, useful for repeating a previous run and using the same data. Training/Validation split is determined randomly.
         """
-        # Build the model
-        base_model = MobileNet(weights='imagenet', include_top=False, input_shape=self.input_shape)
-        x = base_model.output
-        x = GlobalAveragePooling2D()(x)
-        x = Dense(256, activation="relu", name="HiddenLayer1")(x)
-        x = Dropout(0.5)(x)
-        x = Dense(128, activation="relu", name="HiddenLayer2", kernel_regularizer=regularizers.l2(0.01))(x)
-        x = Dropout(0.2)(x)
-        output_layer = Dense(len(DATA_LABELS), activation="sigmoid", name="OutputLayer")(x)
-        self.model = Model(inputs=base_model.input, outputs=output_layer)
+        # # Build the model
+        self.model = tf.keras.Sequential()
+        pretrained_layer = tf.keras.applications.mobilenet.MobileNet(weights='imagenet', include_top=False, input_shape=self.input_shape)
+        self.model.add(pretrained_layer)
+        self.model.add(tf.keras.layers.GlobalAveragePooling2D())
+        self.model.add(tf.keras.layers.Dense(256, activation="relu"))
+        self.model.add(tf.keras.layers.Dropout(0.5))
+        self.model.add(tf.keras.layers.Dense(128, activation="relu"))
+        self.model.add(tf.keras.layers.Dropout(0.2))
+        self.model.add(tf.keras.layers.Dense(len(DATA_LABELS), activation="sigmoid"))
 
-        # for layer in base_model.layers:
-        #     layer.trainable = False
+        self.model.compile(optimizer=tf.keras.optimizers.Adam(0.0005),
+                           loss="binary_crossentropy",
+                           metrics=["accuracy"])
 
-        adam = Adam(lr=0.0005)
-        self.model.compile(optimizer=adam, loss="binary_crossentropy", metrics=["accuracy"])
+        if weights_file is not None and os.path.isfile(weights_file):
+            try:
+                self.model.load_weights(weights_file)
+            except OSError:
+                print("ERROR: Unable to open weights file '{}'".foramt(weights_file))
+                sys.exit(-1)
+            print("Loaded model from file.")
+        else:
+            if weights_file is not None:
+                raise FileNotFoundError
+            print("WARN: No model loaded from file. Generating random model")
 
         if print_summary:
             print(self.model.summary())
 
+        self.quiet = quiet
+        self.seed = seed
+
+    def _init_labels(self):
         # Pull out our labels for use in generators later
         data = pd.read_csv("labels.csv")
         self.training_labels = data.loc[data['evaluation'] == False]  # noqa: E712
@@ -86,27 +80,14 @@ class EyeballModel:
 
         # Shuffle the training labels
         self.random_seed = False
-        self.seed = seed
         if self.seed is None:
             self.random_seed = True
             self.seed = random.randint(0, 999999)
             print("No seed set, ", end='')
-        print(f"using seed: {self.seed}")
+        print("using seed: {}".format(self.seed))
         random.seed(self.seed)
         self.training_labels = self.training_labels.sample(frac=1)
 
-        if weights_file is not None and os.path.isfile(weights_file):
-            try:
-                self.model.load_weights(weights_file)
-            except OSError as e:
-                print(f"ERROR: Unable to open weights file '{weights_file}'")
-                sys.exit(-1)
-            print("Loaded model from file.")
-        else:
-            if weights_file is not None:
-                raise FileNotFoundError
-            print("WARN: No model loaded from file. Generating random model")
-
         # Data augmentation
         augmentor = Augmentor.Pipeline()
         augmentor.set_seed(self.seed)
@@ -131,7 +112,8 @@ class EyeballModel:
         """
         print("Training with seed: " + str(self.seed))
 
-        data_generator = ImageDataGenerator(
+        self._init_labels()
+        data_generator = tf.keras.preprocessing.image.ImageDataGenerator(
             preprocessing_function=self.preprocess_training_function,
             validation_split=0.2,
             samplewise_center=True)
@@ -146,7 +128,7 @@ class EyeballModel:
             subset='training',
             shuffle=True,
             seed=self.seed,
-            class_mode="other")
+            class_mode="raw")
         validation_generator = data_generator.flow_from_dataframe(
             self.training_labels,
             directory=self.image_dir,
@@ -157,13 +139,26 @@ class EyeballModel:
             subset='validation',
             shuffle=False,
             seed=self.seed,
-            class_mode="other")
+            class_mode="raw")
 
         # Model checkpoint - Saves model weights when validation accuracy improves
-        callbacks = [ModelCheckpoint(self.checkpoint_file, monitor='val_loss', verbose=1, save_best_only=True,
-                                     save_weights_only=True, mode='min')]
+        callbacks = [tf.keras.callbacks.ModelCheckpoint(self.checkpoint_file,
+                                                        monitor='val_loss',
+                                                        verbose=1,
+                                                        save_best_only=True,
+                                                        save_weights_only=True,
+                                                        mode='min'),
+                     tf.keras.callbacks.TensorBoard(log_dir='logs',
+                                                    histogram_freq=2,
+                                                    write_graph=True,
+                                                    write_images=False,
+                                                    update_freq='epoch',
+                                                    profile_batch=2,
+                                                    embeddings_freq=0,
+                                                    embeddings_metadata=None),
+                     ]
 
-        history = self.model.fit_generator(
+        history = self.model.fit(
             training_generator,
             steps_per_epoch=len(training_generator.filenames) // batch_size,
             epochs=epochs,
@@ -196,11 +191,30 @@ class EyeballModel:
             plt.legend(['Train', 'Validation'], loc='upper left')
             plt.savefig(self.graphs_directory + "loss.png")
 
-    def predict(self, path):
-        """Predict the labels for a single file
+    def predict_on_array(self, image):
+        """Predict the labels for a single screenshot
 
         Keyword arguments:
-        path -- The path to the file that we'll be evaluating
+        image -- The numpy array of the image to classify
+        """
+        img = np.expand_dims(image, axis=0)
+        img = preprocess_input(img)
+
+        prediction = self.model.predict(img, batch_size=1)
+        result = dict()
+        result["filename"] = "custom-image"
+        result["custom404"] = prediction[0][0]
+        result["login"] = prediction[0][1]
+        result["webapp"] = prediction[0][2]
+        result["oldlooking"] = prediction[0][3]
+        result["parked"] = prediction[0][4]
+        return result
+
+    def predict(self, path, threshold=0.5):
+        """Predict the labels for a single file or directory of files
+
+        Keyword arguments:
+        path -- The path to the file(s) that we'll be evaluating.
         """
         # Is this a single file, or a directory?
         screenshots = []
@@ -210,25 +224,36 @@ class EyeballModel:
             screenshots = os.listdir(path)
             screenshots = [os.path.join(path, s) for s in screenshots]
         else:
-            return None
+            raise FileNotFoundError
 
         results = []
-        for screenshot in screenshots:
+        bar = progressbar.ProgressBar()
+        if self.quiet:
+            bar = progressbar.NullBar()
+        for screenshot in bar(screenshots):
             # Load the image into memory
-            img = image.load_img(screenshot, target_size=(self.image_width, self.image_height))
-            img = image.img_to_array(img)
-            img = np.expand_dims(img, axis=0)
-            img = preprocess_input(img)
+            img = None
+            try:
+                img = tf.keras.preprocessing.image.load_img(screenshot, target_size=(self.image_width, self.image_height))
+                img = tf.keras.preprocessing.image.img_to_array(img)
+                img = np.expand_dims(img, axis=0)
+                img = preprocess_input(img)
+            except IsADirectoryError:
+                print("\nWARN: Skipping directory: ", screenshot)
+                continue
+            except OSError:
+                print("\nWARN: Skipping empty or corrupt file: ", screenshot)
+                continue
 
             prediction = self.model.predict(img, batch_size=1)
             result = dict()
             result["filename"] = screenshot
             result["custom404"] = prediction[0][0]
             result["login"] = prediction[0][1]
-            result["homepage"] = prediction[0][2]
+            result["webapp"] = prediction[0][2]
             result["oldlooking"] = prediction[0][3]
+            result["parked"] = prediction[0][4]
             results.append(result)
-
         return results
 
     def evaluate(self, threshold=0.5):
@@ -238,7 +263,8 @@ class EyeballModel:
         threshold -- Value between 0->1. The cutoff where the numerical prediction becomes boolean. Default: 0.5
         """
         # Prepare the data
-        data_generator = ImageDataGenerator(
+        self._init_labels()
+        data_generator = tf.keras.preprocessing.image.ImageDataGenerator(
             preprocessing_function=preprocess_input)
         evaluation_generator = data_generator.flow_from_dataframe(
             self.evaluation_labels,
@@ -248,12 +274,12 @@ class EyeballModel:
             target_size=(self.image_width, self.image_height),
             shuffle=False,
             batch_size=1,
-            class_mode="other")
+            class_mode="raw")
         # If a seed was selected, then also evaluate on the validation set for that seed
         if not self.random_seed:
             print("Using validation set...")
             # Data augmentation
-            data_generator = ImageDataGenerator(
+            data_generator = tf.keras.preprocessing.image.ImageDataGenerator(
                 preprocessing_function=self.preprocess_training_function,
                 samplewise_center=True,
                 validation_split=0.2)
@@ -267,48 +293,74 @@ class EyeballModel:
                 subset='validation',
                 shuffle=False,
                 seed=self.seed,
-                class_mode="other")
+                class_mode="raw")
         else:
             print("Using evaluation set...")
 
-        prediction_manager = PredictionManager()
-
-        try:
-            self.model.predict_generator(
-                evaluation_generator,
-                callbacks=[prediction_manager],
-                verbose=1,
-                steps=len(evaluation_generator))
-        except KeyboardInterrupt:
-            print('\r\nJob interrupted (Ctrl+C): Outputting completed results')
-
-        predictions = prediction_manager.getPredictions()
-
-        # Truncates evaluation generator if test was interrupted
-        evaluation_generator._data = evaluation_generator.data[:predictions.shape[0]]
+        predictions = self.model.predict(
+            evaluation_generator,
+            verbose=1,
+            steps=len(evaluation_generator))
 
         self._save_prediction_histograms(predictions)
         predictions = predictions > threshold
-        ground_truth = evaluation_generator.data
+        ground_truth = self.evaluation_labels[DATA_LABELS].to_numpy()
+        filenames = self.evaluation_labels[["filename"]].to_numpy()
         stats = classification_report(ground_truth, predictions, target_names=DATA_LABELS, output_dict=True)
         stats["total_binary_accuracy"] = 1 - hamming_loss(ground_truth, predictions)
         stats["all_or_nothing_accuracy"] = accuracy_score(ground_truth, predictions)
-        stats["top_10_best"] = self._top_images(evaluation_generator, predictions, best=True)
-        stats["top_10_worst"] = self._top_images(evaluation_generator, predictions, best=False)
+        stats["top_10_best"] = self._top_images(filenames, ground_truth, predictions, best=True)
+        stats["top_10_worst"] = self._top_images(filenames, ground_truth, predictions, best=False)
+        stats["none_of_the_above_recall"] = self._none_of_the_above_recall(ground_truth, predictions)
+        stats["none_of_the_above_precision"] = self._none_of_the_above_precision(ground_truth, predictions)
         return stats
 
-    def _top_images(self, labels, predictions, top_k=10, best=False):
+    def _none_of_the_above_recall(self, labels, predictions):
+        """Returns the recall score for the 'none of the above' images.
+        That means, all the images that don't have a category.
+        """
+        total_count = 0
+        correct_count = 0
+        for item in zip(labels.astype(int), predictions.astype(int)):
+            # Is this a none of the above?
+            if not item[0].any():
+                total_count += 1
+                if not item[1].any():
+                    correct_count += 1
+        if total_count == 0:
+            print("WARNING: None of the Above Recall is NaN")
+            return 0
+        return correct_count / total_count
+
+    def _none_of_the_above_precision(self, labels, predictions):
+        """Returns the precision score for the 'none of the above' images.
+        That means, all the images that don't have a category.
+        """
+        total_count = 0
+        correct_count = 0
+        for item in zip(labels.astype(int), predictions.astype(int)):
+            # Is this a none of the above prediction?
+            if not item[1].any():
+                total_count += 1
+                if not item[0].any():
+                    correct_count += 1
+        if total_count == 0:
+            print("WARNING: None of the Above Precision is NaN")
+            return 0
+        return correct_count / total_count
+
+    def _top_images(self, filenames, ground_truth, predictions, top_k=10, best=False):
         """Collect top-k best or top-k worst predicted images
 
         Keyword arguments:
-        labels -- The keras generator that contains the filenames and data labels
+        ground_truth -- The correct labels
         predictions -- The numpy array of predictions
         top_k -- Top k elements
         best -- True/False. Calculate either the best or worst images
 
         :Return -- Tuple of top-k indicies and top-k filenames
         """
-        true_labels = np.array(labels.data).astype(float)
+        true_labels = np.array(ground_truth).astype(float)
         predictions = predictions.astype(float)
 
         differences = np.abs(predictions - true_labels).sum(axis=1)
@@ -321,7 +373,7 @@ class EyeballModel:
             indicies = np.flipud(indicies)
 
         for i in indicies[:top_k]:
-            top_file_list.append(labels.filenames[i])
+            top_file_list.append(filenames[i][0])
 
         return indicies[:top_k], top_file_list
 

eyeballer/visualization.py

@@ -0,0 +1,129 @@
+import numpy as np
+import math
+import tensorflow as tf
+
+from copy import deepcopy
+from matplotlib import pyplot as plt
+from eyeballer.model import DATA_LABELS
+
+
+class HeatMap():
+    def __init__(self, screenshot_path, model, threshold=0.5, boxsize=28, step=7):
+        self.model = model
+
+        img = tf.keras.preprocessing.image.load_img(screenshot_path, target_size=(self.model.image_width, self.model.image_height))
+        img = tf.keras.preprocessing.image.img_to_array(img)
+
+        self.screenshot = img
+        self.boxsize = boxsize
+        self.step = step
+        self.x = 0
+        self.y = 0
+        self.screenshot_path = screenshot_path
+        self.gamma = 3
+        self.threshold = threshold
+
+    def generate(self, output_file="heatmap.png"):
+        """ Make a single heatmap image and return it """
+        heatmaps = []
+        labels = []
+        results = self.model.predict(self.screenshot_path)
+        results = results[0]
+        for label in DATA_LABELS:
+            boxsize = self.boxsize
+            # Ignore this label if it didn't predict positively (true label)
+            if results[label] > self.threshold:
+                worst_score = 1
+                while worst_score > self.threshold:
+                    heatmap, worst_score = self._get_heatmap(label, boxsize)
+                    if worst_score > self.threshold:
+                        boxsize += 28
+                        print("Didn't get a good image for {}. Trying again with a bigger boxsize: {}".format(label, boxsize))
+                    else:
+                        heatmaps.append(heatmap)
+                        labels.append(label)
+
+        plt.figure()
+        screenshot_image = tf.keras.preprocessing.image.load_img(self.screenshot_path, target_size=(self.model.image_width, self.model.image_width))
+        _, subplots = plt.subplots(1, len(heatmaps))
+        # This is a little janky, but matplotlib returns a list above if there's multiple subplots, and just a single subplot if there's only one
+        #   it would have been easier if it was always a list. But alas
+        if not heatmaps:
+            print("No heatmap made. The image did not have a True classification for any label")
+            return
+        if len(heatmaps) > 1:
+            for i, heatmap in enumerate(heatmaps):
+                subplots[i].imshow(screenshot_image, cmap='binary', interpolation='none')
+                subplots[i].imshow(heatmap, alpha=0.5, interpolation='none')
+                subplots[i].set_title(labels[i])
+        else:
+            subplots.imshow(screenshot_image, cmap='binary', interpolation='none')
+            subplots.imshow(heatmap, alpha=0.5, interpolation='none')
+            subplots.set_title(labels[0])
+        plt.savefig(output_file)
+        print("Heatmap image written to: {}".format(output_file))
+
+    def _get_heatmap(self, label, boxsize):
+        worst_score = 1
+        self.x, self.y = 0, 0
+
+        heatmap = np.ones((self.model.image_width, self.model.image_height))
+        heatmap *= 255
+        while True:
+            # Occlude an image
+            occluded_image, x, y = self._occlude(boxsize)
+            if occluded_image is not None:
+                # Score the new occluded image
+                results = self.model.predict_on_array(occluded_image)
+                score = results[label]  # TODO
+                worst_score = min(worst_score, score)
+                # Scale the score up to pixel values
+                score = math.floor(score * 255)
+                # Anneal the scores as you go outward into the occlusion block
+                new_scores = np.ones((boxsize, boxsize))
+                new_scores *= score
+                new_scores = self._gamma_anneal(new_scores)
+
+                # For each pixel, update only where the new score is lower
+                original_scores = heatmap[x: x+boxsize, y: y+boxsize]
+
+                target_area = np.where(new_scores < original_scores, new_scores, original_scores)
+                heatmap[x: x+boxsize, y: y+boxsize] = target_area
+            else:
+                break
+
+        return heatmap, worst_score
+
+    def _gamma_anneal(self, occlusion_area):
+        """ Get the annealed score based on centeredness of the occlusion area """
+        width, height = occlusion_area.shape
+
+        center_x = (width-1) / 2
+        center_y = (height-1) / 2
+
+        annealed_area = np.zeros(occlusion_area.shape)
+
+        for (x, y), score in np.ndenumerate(occlusion_area):
+            distance = math.sqrt((center_x - x)**2 + (center_y - y)**2)
+            annealed_area[x, y] = min(255, score + (self.gamma * distance))
+
+        return annealed_area
+
+    def _occlude(self, boxsize):
+        ''' Return a single occluded image and its location '''
+        if self.x + boxsize > self.screenshot.shape[0]:
+            return None, None, None
+
+        retImg = np.copy(self.screenshot)
+        retImg[self.x:self.x+boxsize, self.y:self.y+boxsize] = 0.0
+
+        old_i = deepcopy(self.x)
+        old_j = deepcopy(self.y)
+
+        # update indices
+        self.y = self.y + self.step
+        if self.y+boxsize > self.screenshot.shape[1]:  # reached end
+            self.y = 0  # reset j
+            self.x = self.x + self.step  # go to next row
+
+        return retImg, old_i, old_j

prediction_output_template.html

@@ -0,0 +1,190 @@
+<!DOCTYPE HTML>
+<meta charset="utf-8"/>
+<html>
+<title>Eyeballer Results</title>
+<body>
+    <div id="btnsCont">
+      <button class="btn all" onclick="filterSelection('all')"> Show all</button>
+      <button class="btn" onclick="filterSelection('custom404')"> 404</button>
+      <button class="btn" onclick="filterSelection('login')"> Login</button>
+      <button class="btn" onclick="filterSelection('webapp')"> Web Application</button>
+      <button class="btn" onclick="filterSelection('oldlooking')"> Old Looking</button>
+      <button class="btn" onclick="filterSelection('parked')"> Parked Domain</button>      
+        <select onchange="numOnPage=this.value; filterSelection([]);" style="float:right; margin:5px; margin-right:30px; height:30px; width:auto;">
+          <option value="25">25</option>
+          <option value="50">50</option>
+          <option value="100">100</option>
+          <option value="250">250</option>
+        </select>
+    </div>
+    <div id="btnsPage">
+    </div>
+    <div id="container">
+    </div>
+    <button onclick="topFunction()" id="topBtn" title="Go to top">Top</button>
+<script>
+    var results = {{jsResults}};
+    var selectedTags = new Array();
+    var btnContainer = document.getElementById("btnsCont");
+    var categoryBtns = btnContainer.getElementsByClassName("btn");
+    var numOnPage = 100;
+    var indexesToDisplay = [];
+    filterSelection([]);
+    for (var i = 0; i < categoryBtns.length; i++) {
+        categoryBtns[i].addEventListener("click", function () {
+            if (this.className.indexOf("active") > -1) {
+                this.className = this.className.replace(" active", "");
+            }
+            else {
+            this.className += " active";
+            }
+        });
+    }
+    function filterSelection(c) {
+        indexesToDisplay = [];
+        var currentIndex = -1;
+        if (c == "all") location.reload();
+        if (selectedTags.includes(c)!=true) selectedTags=selectedTags.concat(c);
+        else selectedTags.splice(selectedTags.indexOf(c),1);
+        for (var filename in results){
+                currentIndex++;
+                var tags = results[filename];
+                var isMissingTag = false;
+                for (var i in selectedTags){
+                    if (!tags.includes(selectedTags[i])){
+                        isMissingTag = true;
+                        break;
+                    }
+                }
+                if (isMissingTag === false){
+                    indexesToDisplay = indexesToDisplay.concat(currentIndex);
+                }
+            }
+        buildPageButtons(indexesToDisplay);
+        buildHTML(0, numOnPage);
+    }
+    function buildPageButtons(indexes){
+        numOfPages = Math.ceil(indexes.length/numOnPage);
+        var html = '';
+        for (var i = 0; i < numOfPages; i++){
+            html += ('<button class="btnpage" onclick="buildHTML(' + (i*numOnPage) + ', ' + ((i+1)*numOnPage) + '); activatePage(this);">' + (i+1) +'</button>');
+        }
+        document.getElementById('btnsPage').innerHTML = html;
+    }
+    function activatePage(pressedBtn){
+        var activeNow = document.getElementsByClassName('activePage')
+        for (var i = 0; i < activeNow.length; i++){
+            activeNow[i].className = activeNow[i].className.replace(" activePage", "");
+        }
+        pressedBtn.className += ' activePage';
+    }
+    function buildHTML(rangeMin, rangeMax){
+        var html = '';
+        if (rangeMax > indexesToDisplay.length) rangeMax=indexesToDisplay.length;
+        for (var i = rangeMin; i < rangeMax; i++){
+            index=indexesToDisplay[i];
+            filename = Object.keys(results)[index];
+            html += ('<div class="filterDiv"><a href=' + filename + ' target="_blank"> ' + results[filename].toString().replace(/,/g, ' ') + '</a><img class="res_pic" src="' + filename + '"></div>');
+        }
+        document.getElementById('container').innerHTML = html;
+    }
+    window.onscroll = function() {scrollFunction()};
+    function scrollFunction() {
+      if (document.body.scrollTop > 20 || document.documentElement.scrollTop > 20) {
+        document.getElementById("topBtn").style.display = "block";
+      } else {
+        document.getElementById("topBtn").style.display = "none";
+      }
+    }
+    function topFunction() {
+      document.body.scrollTop = 0; // For Safari
+      document.documentElement.scrollTop = 0; // For Chrome, Firefox, IE and Opera
+    }
+</script>
+    <style>
+    #topBtn {
+      display: none; /* Hidden by default */
+      position: fixed; /* Fixed/sticky position */
+      bottom: 20px; /* Place the button at the bottom of the page */
+      right: 30px; /* Place the button 30px from the right */
+      z-index: 99; /* Make sure it does not overlap */
+      border: none; /* Remove borders */
+      outline: none; /* Remove outline */
+      background-color: #193b72; /* Set a background color */
+      color: white; /* Text color */
+      cursor: pointer; /* Add a mouse pointer on hover */
+      padding: 15px; /* Some padding */
+      border-radius: 10px; /* Rounded corners */
+      font-size: 18px; /* Increase font size */
+      font-weight:bold;
+      border:solid;
+    }
+    #topBtn:hover {
+      background-color: #555;
+    }
+    .container {
+    overflow: hidden;
+    }
+    #btnsPage {
+    margin:10px;
+    }
+    .btnpage{
+        width:40px;
+        height:30px;
+        background-color: #f1f1f1;
+        outline: none;
+    }
+    .res_pic{
+    width:100%;
+    height:auto;
+    }
+    a{
+    font-weight:600;
+    padding:3px;
+    text-decoration: none;
+    color:#ffffff;
+    font-size:16px;
+    font-family:Sans-serif;
+    }
+    .name{
+    float:left;
+    }
+    .filterDiv {
+      float: left;
+      background-color: #777;
+      color: #ffffff;
+      width: 22%;
+      border-style:solid;
+      border-color:#666;
+      text-align: center;
+      margin: 10px;
+      line-height: 30px;
+      display: block;
+    }
+    .btn {
+      border: none;
+      outline: none;
+      padding: 12px 16px;
+      margin-left:10px;
+      background-color: #f1f1f1;
+      cursor: pointer;
+    }
+    .btn:hover {
+      background-color: #ddd;
+    }
+    .active {
+      background-color: #666;
+      color: white;
+    }
+    .activePage{
+        border: solid;
+        border-color:#666666;
+        color: black;
+    }
+    .btn.all {
+      background-color: #193b72;
+      color: white;
+    }
+    </style>
+</body>
+</html>

requirements-gpu.txt

@@ -0,0 +1,10 @@
+Augmentor
+click
+matplotlib
+numpy
+pandas
+pillow
+sklearn
+tensorflow-gpu
+jinja2
+progressbar2

requirements.txt

@@ -6,4 +6,5 @@ pandas
 pillow
 sklearn
 tensorflow
-git+https://github.com/keras-team/keras/
+jinja2
+progressbar2

tests/PredictTest.py

@@ -0,0 +1,142 @@
+import errno
+import os
+import unittest
+import sys
+
+from eyeballer.model import EyeballModel
+from eyeballer.visualization import HeatMap
+
+
+class PredictTest(unittest.TestCase):
+    def setUp(self):
+
+        class DummyFile(object):
+            def write(self, x): pass
+            def flush(self): pass
+
+        sys.stdout = DummyFile()
+        weights_file = "tests/models/test_weights.h5"
+        if not os.path.isfile(weights_file):
+            print("Error: Symlink the latest weights file to " + weights_file)
+            raise FileNotFoundError(
+                errno.ENOENT,
+                os.strerror(errno.ENOENT),
+                weights_file)
+
+        model_kwargs = {
+            "weights_file": weights_file,
+            "print_summary": False,
+            "seed": None,
+            "quiet": True
+        }
+        self.model = EyeballModel(**model_kwargs)
+
+    def test_different_seed_predict(self):
+        model_kwargs = {
+            "weights_file": None,
+            "print_summary": False,
+            "seed": 12345678,
+            "quiet": True
+        }
+        same_seed_model = EyeballModel(**model_kwargs)
+
+        screenshot = "tests/data/404.png"
+
+        results_one = self.model.predict(screenshot)
+        results_two = same_seed_model.predict(screenshot)
+
+        self.assertNotEqual(results_one, results_two)
+
+    def test_same_seed_predict(self):
+        model_kwargs = {
+            "weights_file": None,
+            "print_summary": False,
+            "seed": 12345678,
+            "quiet": True
+        }
+        same_seed_model = EyeballModel(**model_kwargs)
+
+        screenshot = "tests/data/404.png"
+
+        results_one = same_seed_model.predict(screenshot)
+        results_two = same_seed_model.predict(screenshot)
+
+        self.assertEqual(results_one, results_two)
+
+    def test_predict_custom404(self):
+        screenshot = "tests/data/404.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertGreater(results["custom404"], 0.5)
+
+    def test_predict_not_custom404(self):
+        screenshot = "tests/data/nothing.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertLess(results["custom404"], 0.5)
+
+    def test_predict_login(self):
+        screenshot = "tests/data/login.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertGreater(results["login"], 0.5)
+
+    def test_predict_not_login(self):
+        screenshot = "tests/data/nothing.png"
+        results = self.model.predict(screenshot)[0]
+        print(screenshot, results)
+        self.assertLess(results["login"], 0.5)
+
+    def test_predict_homepage(self):
+        screenshot = "tests/data/homepage.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertGreater(results["homepage"], 0.5)
+
+    def test_predict_not_homepage(self):
+        screenshot = "tests/data/nothing.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertLess(results["homepage"], 0.5)
+
+    def test_predict_oldlooking(self):
+        screenshot = "tests/data/old-looking.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertGreater(results["oldlooking"], 0.5)
+
+    def test_predict_not_oldlooking(self):
+        screenshot = "tests/data/nothing.png"
+        results = self.model.predict(screenshot)[0]
+        self.assertLess(results["oldlooking"], 0.5)
+
+    def test_file_doesnt_exist(self):
+        screenshot = "tests/data/doesnotexist.png"
+        try:
+            self.model.predict(screenshot)[0]
+            self.fail("FileNotFoundError was expected but not found")
+        except FileNotFoundError:
+            pass
+
+    def test_folder(self):
+        screenshots = "tests/data/"
+        results = self.model.predict(screenshots)
+        self.assertEqual(len(results), 7)
+
+    def test_file_is_empty(self):
+        """
+        We're just testing that it doesn't crash, basically
+        """
+        screenshot = "tests/data/empty.png"
+        self.model.predict(screenshot)
+
+    def test_file_is_invalid(self):
+        """
+        We're just testing that it doesn't crash, basically
+        """
+        screenshot = "tests/data/invalid.png"
+        self.model.predict(screenshot)
+
+    def test_heatmap(self):
+        screenshot = "tests/data/login.png"
+        HeatMap(screenshot, self.model, 0.5)
+
+        screenshot = "tests/data/404.png"
+        HeatMap(screenshot, self.model, 0.5)
+
+        screenshot = "tests/data/empty.png"
+        HeatMap(screenshot, self.model, 0.5)

tests/__init__.py

@@ -0,0 +1 @@
+from tests.PredictTest import *  # noqa

tests/data/invalid.png

@@ -0,0 +1 @@
+THIS TEXT IS NOT A VALID PNG

utils/resizer.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
 
 # Converts full-size images to a given size (replacing the originals)
-for f in images_full_size/*
+for f in images/*
 do
-	convert -resize 224x224 $f $f
+	echo $f
+	convert -resize 224x224\! $f $f
 done

utils/verify.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+
+import csv
+import random
+from os import listdir
+from os.path import isfile, join
+
+with open("labels.csv", newline="") as csvfile:
+    # Get the header labels
+    csvreader = csv.DictReader(csvfile)
+    # fieldnames = next(csvreader)
+    file_list = [f for f in listdir("images") if isfile(join("images", f))]
+
+    label_list = []
+    for row in csvreader:
+        label_list.append(row["filename"])
+
+    # Loop through the file
+    print("Rows in labels.csv, but no image exists:")
+    for label in label_list:
+        if label not in file_list:
+            print(row["filename"])
+
+    print("Images that exist, but don't have labels:")
+    i = 0
+    for image in file_list:
+        if image not in label_list:
+            print(image)

webui/.gitignore

@@ -0,0 +1,48 @@
+# See http://help.github.com/ignore-files/ for more about ignoring files.
+
+# compiled output
+/dist
+/tmp
+/out-tsc
+# Only exists if Bazel was run
+/bazel-out
+
+.vscode/
+
+# dependencies
+/node_modules
+
+# profiling files
+chrome-profiler-events*.json
+speed-measure-plugin*.json
+
+# IDEs and editors
+/.idea
+.project
+.classpath
+.c9/
+*.launch
+.settings/
+*.sublime-workspace
+
+# IDE - VSCode
+.vscode/*
+!.vscode/settings.json
+!.vscode/tasks.json
+!.vscode/launch.json
+!.vscode/extensions.json
+.history/*
+
+# misc
+/.sass-cache
+/connect.lock
+/coverage
+/libpeerconnection.log
+npm-debug.log
+yarn-error.log
+testem.log
+/typings
+
+# System Files
+.DS_Store
+Thumbs.db

webui/README.md

@@ -0,0 +1,11 @@
+# Weballer
+
+Web front-end for Eyeballer
+
+
+### Setup
+
+```
+npm install
+ng serve
+```

webui/angular.json

@@ -0,0 +1,133 @@
+{
+  "$schema": "./node_modules/@angular/cli/lib/config/schema.json",
+  "version": 1,
+  "newProjectRoot": "projects",
+  "projects": {
+    "weballer": {
+      "projectType": "application",
+      "schematics": {
+        "@schematics/angular:component": {
+          "style": "scss"
+        }
+      },
+      "root": "",
+      "sourceRoot": "src",
+      "prefix": "app",
+      "architect": {
+        "build": {
+          "builder": "@angular-devkit/build-angular:browser",
+          "options": {
+            "outputPath": "dist/weballer",
+            "index": "src/index.html",
+            "main": "src/main.ts",
+            "polyfills": "src/polyfills.ts",
+            "tsConfig": "tsconfig.app.json",
+            "aot": true,
+            "assets": [
+              "src/favicon.ico",
+              "src/assets"
+            ],
+            "styles": [
+              "./node_modules/@angular/material/prebuilt-themes/deeppurple-amber.css",
+              "./node_modules/@fortawesome/fontawesome-free/css/all.min.css",
+              "src/styles.scss"
+            ],
+            "scripts": []
+          },
+          "configurations": {
+            "production": {
+              "fileReplacements": [
+                {
+                  "replace": "src/environments/environment.ts",
+                  "with": "src/environments/environment.prod.ts"
+                }
+              ],
+              "optimization": true,
+              "outputHashing": "all",
+              "sourceMap": false,
+              "namedChunks": false,
+              "extractLicenses": true,
+              "vendorChunk": false,
+              "buildOptimizer": true,
+              "budgets": [
+                {
+                  "type": "initial",
+                  "maximumWarning": "2mb",
+                  "maximumError": "5mb"
+                },
+                {
+                  "type": "anyComponentStyle",
+                  "maximumWarning": "6kb",
+                  "maximumError": "10kb"
+                }
+              ]
+            }
+          }
+        },
+        "serve": {
+          "builder": "@angular-devkit/build-angular:dev-server",
+          "options": {
+            "browserTarget": "weballer:build"
+          },
+          "configurations": {
+            "production": {
+              "browserTarget": "weballer:build:production"
+            }
+          }
+        },
+        "extract-i18n": {
+          "builder": "@angular-devkit/build-angular:extract-i18n",
+          "options": {
+            "browserTarget": "weballer:build"
+          }
+        },
+        "test": {
+          "builder": "@angular-devkit/build-angular:karma",
+          "options": {
+            "main": "src/test.ts",
+            "polyfills": "src/polyfills.ts",
+            "tsConfig": "tsconfig.spec.json",
+            "karmaConfig": "karma.conf.js",
+            "assets": [
+              "src/favicon.ico",
+              "src/assets"
+            ],
+            "styles": [
+              "src/styles.scss"
+            ],
+            "scripts": []
+          }
+        },
+        "lint": {
+          "builder": "@angular-devkit/build-angular:tslint",
+          "options": {
+            "tsConfig": [
+              "tsconfig.app.json",
+              "tsconfig.spec.json",
+              "e2e/tsconfig.json"
+            ],
+            "exclude": [
+              "**/node_modules/**"
+            ]
+          }
+        },
+        "e2e": {
+          "builder": "@angular-devkit/build-angular:protractor",
+          "options": {
+            "protractorConfig": "e2e/protractor.conf.js",
+            "devServerTarget": "weballer:serve"
+          },
+          "configurations": {
+            "production": {
+              "devServerTarget": "weballer:serve:production"
+            }
+          }
+        }
+      }
+    }
+  },
+  "defaultProject": "weballer",
+  "cli": {
+    "analytics": false
+  }
+}

webui/e2e/protractor.conf.js

@@ -0,0 +1,36 @@
+// @ts-check
+// Protractor configuration file, see link for more information
+// https://github.com/angular/protractor/blob/master/lib/config.ts
+
+const { SpecReporter, StacktraceOption } = require('jasmine-spec-reporter');
+
+/**
+ * @type { import("protractor").Config }
+ */
+exports.config = {
+  allScriptsTimeout: 11000,
+  specs: [
+    './src/**/*.e2e-spec.ts'
+  ],
+  capabilities: {
+    browserName: 'chrome'
+  },
+  directConnect: true,
+  baseUrl: 'http://localhost:4200/',
+  framework: 'jasmine',
+  jasmineNodeOpts: {
+    showColors: true,
+    defaultTimeoutInterval: 30000,
+    print: function() {}
+  },
+  onPrepare() {
+    require('ts-node').register({
+      project: require('path').join(__dirname, './tsconfig.json')
+    });
+    jasmine.getEnv().addReporter(new SpecReporter({
+      spec: {
+        displayStacktrace: StacktraceOption.PRETTY
+      }
+    }));
+  }
+};

webui/e2e/src/app.e2e-spec.ts

@@ -0,0 +1,23 @@
+import { AppPage } from './app.po';
+import { browser, logging } from 'protractor';
+
+describe('workspace-project App', () => {
+  let page: AppPage;
+
+  beforeEach(() => {
+    page = new AppPage();
+  });
+
+  it('should display welcome message', () => {
+    page.navigateTo();
+    expect(page.getTitleText()).toEqual('weballer app is running!');
+  });
+
+  afterEach(async () => {
+    // Assert that there are no errors emitted from the browser
+    const logs = await browser.manage().logs().get(logging.Type.BROWSER);
+    expect(logs).not.toContain(jasmine.objectContaining({
+      level: logging.Level.SEVERE,
+    } as logging.Entry));
+  });
+});

webui/e2e/src/app.po.ts

@@ -0,0 +1,11 @@
+import { browser, by, element } from 'protractor';
+
+export class AppPage {
+  navigateTo(): Promise<unknown> {
+    return browser.get(browser.baseUrl) as Promise<unknown>;
+  }
+
+  getTitleText(): Promise<string> {
+    return element(by.css('app-root .content span')).getText() as Promise<string>;
+  }
+}

webui/e2e/tsconfig.json

@@ -0,0 +1,14 @@
+/* To learn more about this file see: https://angular.io/config/tsconfig. */
+{
+  "extends": "../tsconfig.json",
+  "compilerOptions": {
+    "outDir": "../out-tsc/e2e",
+    "module": "commonjs",
+    "target": "es2018",
+    "types": [
+      "jasmine",
+      "jasminewd2",
+      "node"
+    ]
+  }
+}

webui/karma.conf.js

@@ -0,0 +1,32 @@
+// Karma configuration file, see link for more information
+// https://karma-runner.github.io/1.0/config/configuration-file.html
+
+module.exports = function (config) {
+  config.set({
+    basePath: '',
+    frameworks: ['jasmine', '@angular-devkit/build-angular'],
+    plugins: [
+      require('karma-jasmine'),
+      require('karma-chrome-launcher'),
+      require('karma-jasmine-html-reporter'),
+      require('karma-coverage-istanbul-reporter'),
+      require('@angular-devkit/build-angular/plugins/karma')
+    ],
+    client: {
+      clearContext: false // leave Jasmine Spec Runner output visible in browser
+    },
+    coverageIstanbulReporter: {
+      dir: require('path').join(__dirname, './coverage/weballer'),
+      reports: ['html', 'lcovonly', 'text-summary'],
+      fixWebpackSourcePaths: true
+    },
+    reporters: ['progress', 'kjhtml'],
+    port: 9876,
+    colors: true,
+    logLevel: config.LOG_INFO,
+    autoWatch: true,
+    browsers: ['Chrome'],
+    singleRun: false,
+    restartOnFileChange: true
+  });
+};

webui/package.json

@@ -0,0 +1,54 @@
+{
+  "name": "weballer",
+  "version": "0.0.0",
+  "scripts": {
+    "ng": "ng",
+    "start": "ng serve",
+    "build": "ng build",
+    "test": "ng test",
+    "lint": "ng lint",
+    "e2e": "ng e2e"
+  },
+  "private": true,
+  "dependencies": {
+    "@angular/animations": "~11.2.6",
+    "@angular/cdk": "^11.2.5",
+    "@angular/common": "~11.2.6",
+    "@angular/compiler": "~11.2.6",
+    "@angular/core": "~11.2.6",
+    "@angular/flex-layout": "^11.0.0-beta.33",
+    "@angular/forms": "~11.2.6",
+    "@angular/material": "^11.2.5",
+    "@angular/platform-browser": "~11.2.6",
+    "@angular/platform-browser-dynamic": "~11.2.6",
+    "@angular/router": "~11.2.6",
+    "@fortawesome/fontawesome-free": "^5.15.3",
+    "@tensorflow/tfjs": "^3.3.0",
+    "base64-arraybuffer": "^0.2.0",
+    "ngx-dropzone": "^3.0.0",
+    "rxjs": "~6.6.6",
+    "tslib": "^2.0.0",
+    "zone.js": "~0.10.3"
+  },
+  "devDependencies": {
+    "@angular-devkit/build-angular": "^0.1102.5",
+    "@angular/cli": "~11.2.5",
+    "@angular/compiler-cli": "~11.2.6",
+    "@types/jasmine": "~3.6.0",
+    "@types/jasminewd2": "~2.0.3",
+    "@types/node": "^12.11.1",
+    "@types/webgl2": "0.0.6",
+    "codelyzer": "^6.0.0",
+    "jasmine-core": "~3.6.0",
+    "jasmine-spec-reporter": "~5.0.0",
+    "karma": "~6.2.0",
+    "karma-chrome-launcher": "~3.1.0",
+    "karma-coverage-istanbul-reporter": "~3.0.2",
+    "karma-jasmine": "~4.0.0",
+    "karma-jasmine-html-reporter": "^1.5.0",
+    "protractor": "~7.0.0",
+    "ts-node": "~8.3.0",
+    "tslint": "~6.1.0",
+    "typescript": "~4.1.5"
+  }
+}

webui/src/app/app-routing.module.ts

@@ -0,0 +1,13 @@
+import { NgModule } from '@angular/core';
+import { Routes, RouterModule } from '@angular/router';
+import { EyeballerComponent } from './components/eyeballer/eyeballer.component';
+
+const routes: Routes = [
+  { path: '', component: EyeballerComponent },
+];
+
+@NgModule({
+  imports: [RouterModule.forRoot(routes, { relativeLinkResolution: 'legacy' })],
+  exports: [RouterModule]
+})
+export class AppRoutingModule { }

webui/src/app/app.component.html

@@ -0,0 +1,2 @@
+<router-outlet>
+<router-outlet>

webui/src/app/app.component.spec.ts

@@ -0,0 +1,35 @@
+import { TestBed, waitForAsync } from '@angular/core/testing';
+import { RouterTestingModule } from '@angular/router/testing';
+import { AppComponent } from './app.component';
+
+describe('AppComponent', () => {
+  beforeEach(waitForAsync(() => {
+    TestBed.configureTestingModule({
+      imports: [
+        RouterTestingModule
+      ],
+      declarations: [
+        AppComponent
+      ],
+    }).compileComponents();
+  }));
+
+  it('should create the app', () => {
+    const fixture = TestBed.createComponent(AppComponent);
+    const app = fixture.componentInstance;
+    expect(app).toBeTruthy();
+  });
+
+  it(`should have as title 'weballer'`, () => {
+    const fixture = TestBed.createComponent(AppComponent);
+    const app = fixture.componentInstance;
+    expect(app.title).toEqual('weballer');
+  });
+
+  it('should render title', () => {
+    const fixture = TestBed.createComponent(AppComponent);
+    fixture.detectChanges();
+    const compiled = fixture.nativeElement;
+    expect(compiled.querySelector('.content span').textContent).toContain('weballer app is running!');
+  });
+});

webui/src/app/app.component.ts

@@ -0,0 +1,10 @@
+import { Component } from '@angular/core';
+
+@Component({
+  selector: 'app-root',
+  templateUrl: './app.component.html',
+  styleUrls: ['./app.component.scss']
+})
+export class AppComponent {
+  title = 'weballer';
+}

webui/src/app/app.module.ts

@@ -0,0 +1,28 @@
+import { BrowserModule } from '@angular/platform-browser';
+import { NgModule } from '@angular/core';
+import { FlexLayoutModule } from "@angular/flex-layout";
+import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
+import { NgxDropzoneModule } from 'ngx-dropzone';
+
+import { BaseMaterialModule } from './base-materials';
+import { AppRoutingModule } from './app-routing.module';
+import { AppComponent } from './app.component';
+import { EyeballerComponent } from './components/eyeballer/eyeballer.component';
+
+@NgModule({
+  declarations: [
+    AppComponent,
+    EyeballerComponent
+  ],
+  imports: [
+    BrowserModule,
+    AppRoutingModule,
+    NgxDropzoneModule,
+    BrowserAnimationsModule,
+    BaseMaterialModule,
+    FlexLayoutModule
+  ],
+  providers: [],
+  bootstrap: [AppComponent]
+})
+export class AppModule { }

webui/src/app/base-materials.ts

@@ -0,0 +1,58 @@
+/*
+  Electric Scan
+  Copyright (C) 2019  Bishop Fox
+
+  This program is free software; you can redistribute it and/or
+  modify it under the terms of the GNU General Public License
+  as published by the Free Software Foundation; either version 2
+  of the License, or (at your option) any later version.
+
+  This program is distributed in the hope that it will be useful,
+  but WITHOUT ANY WARRANTY; without even the implied warranty of
+  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+  GNU General Public License for more details.
+
+  You should have received a copy of the GNU General Public License
+  along with this program; if not, write to the Free Software
+  Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+*/
+
+import { NgModule } from '@angular/core';
+import { MatBadgeModule } from '@angular/material/badge';
+import { MatButtonModule } from '@angular/material/button';
+import { MatCardModule } from '@angular/material/card';
+import { MatCheckboxModule } from '@angular/material/checkbox';
+import { MatOptionModule } from '@angular/material/core';
+import { MatDialogModule } from '@angular/material/dialog';
+import { MatDividerModule } from '@angular/material/divider';
+import { MatFormFieldModule } from '@angular/material/form-field';
+import { MatGridListModule } from '@angular/material/grid-list';
+import { MatIconModule } from '@angular/material/icon';
+import { MatInputModule } from '@angular/material/input';
+import { MatMenuModule } from '@angular/material/menu';
+import { MatProgressBarModule } from '@angular/material/progress-bar';
+import { MatProgressSpinnerModule } from '@angular/material/progress-spinner';
+import { MatSelectModule } from '@angular/material/select';
+import { MatSidenavModule } from '@angular/material/sidenav';
+import { MatSlideToggleModule } from '@angular/material/slide-toggle';
+import { MatSnackBarModule } from '@angular/material/snack-bar';
+import { MatSortModule } from '@angular/material/sort';
+import { MatTableModule } from '@angular/material/table';
+import { MatTabsModule } from '@angular/material/tabs';
+import { MatToolbarModule } from '@angular/material/toolbar';
+import { MatTooltipModule } from '@angular/material/tooltip';
+import { MatExpansionModule } from '@angular/material/expansion';
+
+const modules = [
+  MatButtonModule, MatCheckboxModule, MatOptionModule, MatSelectModule, MatMenuModule, MatTabsModule,
+  MatFormFieldModule, MatTooltipModule, MatToolbarModule, MatIconModule, MatProgressSpinnerModule,
+  MatDialogModule, MatGridListModule, MatCardModule, MatTableModule, MatSortModule, MatInputModule,
+  MatSnackBarModule, MatSlideToggleModule, MatDividerModule, MatProgressBarModule, MatBadgeModule,
+  MatSidenavModule, MatExpansionModule
+];
+
+@NgModule({
+  imports: modules,
+  exports: modules,
+})
+export class BaseMaterialModule { }

webui/src/app/components/eyeballer/eyeballer.component.html

@@ -0,0 +1,114 @@
+<mat-toolbar class="app-toolbar" color="primary"  fxLayout="row" fxLayoutAlign="start center">
+  <h2>
+    <img src="assets/eyeballer_logo.png" width="46" height="60">
+  </h2>
+  Eyeballer
+  <span fxFlex></span>
+  <span>Bishop Fox R&D</span>
+  <span fxFlex></span>
+
+  <button mat-raised-button (click)="restart()">
+    <i class="fas fa-fw fa-undo"></i>
+    Restart
+  </button>
+
+</mat-toolbar>
+
+<!-- Model Loading Progress -->
+<div *ngIf="!tfFilesCompleted">
+  <mat-progress-bar mode="indeterminate"></mat-progress-bar>
+</div>
+
+<!-- Drop Zone -->
+<div *ngIf="tfFilesCompleted && !eyeballing && !eyeballCompleted">
+  <mat-card>
+    Drop your screenshots below:
+    <br><br>
+    <ngx-dropzone (change)="onSelect($event)">
+      <ngx-dropzone-label>
+        Files dropped here are <span style="font-weight: bold">not uploaded anywhere.</span> No data leaves your computer at all on this tool.
+      </ngx-dropzone-label>
+    </ngx-dropzone>
+  </mat-card>
+
+  <br />
+
+  <mat-card><span style="font-weight: bold">Note:</span> For best results,
+    ensure your screenshots are taken natively in a <span style="font-weight: bold">16:10 aspect ratio.</span> (ie: 1440x900 or other)
+  </mat-card>
+  <mat-card>
+    Source code available at <a href="https://github.com/BishopFox/eyeballer" target="_blank" rel="noopener noreferrer">GitHub</a>
+  </mat-card>
+</div>
+
+<!-- Progress Bar -->
+<div *ngIf="tfFilesCompleted && eyeballing && !eyeballCompleted">
+  <mat-card>
+    <h3>Classifying: {{this.eyeballedCount}} of {{this.imageCount}}</h3>
+    <mat-progress-bar mode="determinate" [value]="eyeballPercent()"></mat-progress-bar>
+  </mat-card>
+  <mat-card>
+    This processing is happening in your browser. <span style="font-weight: bold">There is no backend.</span>
+  </mat-card>
+</div>
+
+<!-- Results -->
+<div *ngIf="tfFilesCompleted && !eyeballing && eyeballCompleted">
+  <div class="results-container">
+    <h2>Results</h2>
+    <mat-toolbar>
+      <mat-toolbar-row style="height:40px;">
+        <span class="button-spacer"></span>
+        <button class="label-button" mat-raised-button color="primary" [disabled]="selected.webapp === 0" (click)="selected.webapp = 0; updateSelections()">Must Include</button>
+        <button class="label-button" mat-raised-button color="basic" [disabled]="selected.webapp === 1" (click)="selected.webapp = 1; updateSelections()">Webapp</button>
+        <button class="label-button" mat-raised-button color="accent" [disabled]="selected.webapp === 2" (click)="selected.webapp = 2; updateSelections()">Never Include</button>
+        <label class="label-count">{{ classifications.webapp.length }}</label>
+        <span class="button-spacer"></span>
+      </mat-toolbar-row>
+      <mat-toolbar-row style="height:40px;">
+        <span class="button-spacer"></span>
+        <button class="label-button" mat-raised-button color="primary" [disabled]="selected.oldLooking === 0" (click)="selected.oldLooking = 0; updateSelections()">Must Include</button>
+        <button class="label-button" mat-raised-button color="basic" [disabled]="selected.oldLooking === 1" (click)="selected.oldLooking = 1; updateSelections()">Old Looking</button>
+        <button class="label-button" mat-raised-button color="accent" [disabled]="selected.oldLooking === 2" (click)="selected.oldLooking = 2; updateSelections()">Never Include</button>
+        <label class="label-count">{{ classifications.oldLooking.length }}</label>
+        <span class="button-spacer"></span>
+      </mat-toolbar-row>
+      <mat-toolbar-row style="height:40px;">
+        <span class="button-spacer"></span>
+        <button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.loginPage === 0" (click)="selected.loginPage = 0; updateSelections()">Must Include</button>
+        <button class="label-button" mat-raised-button color="basic" [disabled]="selected.loginPage === 1" (click)="selected.loginPage = 1; updateSelections()">Login Page</button>
+        <button class="label-button" mat-raised-button color="accent" [disabled]="selected.loginPage === 2" (click)="selected.loginPage = 2; updateSelections()">Never Include</button>
+        <label class="label-count">{{ classifications.loginPage.length }}</label>
+        <span class="button-spacer"></span>
+      </mat-toolbar-row>
+      <mat-toolbar-row style="height:40px;">
+        <span class="button-spacer"></span>
+        <button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.custom404 === 0" (click)="selected.custom404 = 0; updateSelections()">Must Include</button>
+        <button class="label-button" mat-raised-button color="basic" [disabled]="selected.custom404 === 1" (click)="selected.custom404 = 1; updateSelections()">Custom 404</button>
+        <button class="label-button" mat-raised-button color="accent" [disabled]="selected.custom404 === 2" (click)="selected.custom404 = 2; updateSelections()">Never Include</button>
+        <label class="label-count">{{ classifications.custom404.length }}</label>
+        <span class="button-spacer"></span>
+      </mat-toolbar-row>
+      <mat-toolbar-row style="height:40px;">
+        <span class="button-spacer"></span>
+        <button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.parked === 0" (click)="selected.parked = 0; updateSelections()">Must Include</button>
+        <button class="label-button" mat-raised-button color="basic" [disabled]="selected.parked === 1" (click)="selected.parked = 1; updateSelections()">Parked Domain</button>
+        <button class="label-button" mat-raised-button color="accent" [disabled]="selected.parked === 2" (click)="selected.parked = 2; updateSelections()">Never Include</button>
+        <label class="label-count">{{ classifications.parked.length }}</label>
+        <span class="button-spacer"></span>
+      </mat-toolbar-row>
+    </mat-toolbar>
+
+    <mat-card>
+      <span style="font-weight: bold">Total Selected:</span> {{ selectedScreens.length }}
+    </mat-card>
+    <br>
+    <mat-card>
+      <button class="export-button" mat-raised-button color="primary" (click)="exportResults()">
+        Export
+      </button>
+      <a href="https://github.com/BishopFox/eyeballer#what-the-labels-mean" target="_blank" rel="noopener noreferrer">What do these labels mean?</a>
+    </mat-card>
+  </div>
+
+</div>

webui/src/app/components/eyeballer/eyeballer.component.scss

@@ -0,0 +1,128 @@
+.app-toolbar {
+    position: fixed;
+    position: -webkit-sticky; /* For macOS/iOS Safari */
+    top: 0; /* Sets the sticky toolbar to be on top */
+    z-index: 1000; /* Ensure that your app's content doesn't overlap the toolbar */
+}
+
+.not-completed-container {
+    display: grid;
+    justify-items: center;
+    align-items: center;
+
+    grid-template-columns: 1fr 1fr 1fr;
+}
+
+.not-completed-message {
+    grid-column: 2/3;
+    grid-row: 2/3;
+}
+
+.results-container {
+    display: grid;
+    justify-items: center;
+    align-items: center;
+
+    grid-template-columns: 1fr;
+}
+
+.eyeballed-class-container {
+    display: grid;
+    justify-items: center;
+    align-items: center;
+
+    // grid-template-rows: minmax(250px, auto);
+    grid-template-columns: 1fr 1fr 1fr 1fr;
+    grid-gap: 1em;
+    margin: 1em;
+}
+
+.result-card {
+    max-width: 100%;
+}
+
+.result-card:hover {
+    background-color: #f5f5f5;
+}
+
+.result-card:hover img {
+    opacity: .5;
+}
+
+.result-card > div {
+    display: none;
+    position: absolute;
+    top: 50%;
+    left: 50%;
+    width: 70px;
+    height: 70px;
+    z-index: 10;
+}
+
+.fill-remaining-space {
+    flex: 1 1 auto;
+}
+
+.fill-space {
+    flex: auto;
+}
+
+.img-container {
+  position: relative;
+  text-align: center;
+  color: white;
+}
+
+.preview-image {
+  width: 100%;
+}
+
+.webapp-label {
+  background-color: blue;
+  color: white;
+  margin-left: 1px;
+  margin-right: 1px;
+}
+.old-looking-label {
+  background-color: green;
+  color: white;
+  margin-left: 1px;
+  margin-right: 1px;
+}
+.login-label {
+  background-color: purple;
+  color: white;
+  margin-left: 1px;
+  margin-right: 1px;
+}
+.custom-404-label {
+  background-color: orange;
+  color: white;
+  margin-left: 1px;
+  margin-right: 1px;
+}
+.parked-label {
+  background-color: red;
+  color: white;
+  margin-left: 1px;
+  margin-right: 1px;
+}
+
+.label-count {
+  margin-left: 10px;
+  width: 50px!important;
+  min-width: unset!important;
+}
+
+.label-button{
+  width: 130px!important;
+  min-width: unset!important;
+}
+
+.button-spacer {
+  flex: 1 1 auto;
+}
+
+.export-button {
+  margin-right: 10px;
+}

webui/src/app/components/eyeballer/eyeballer.component.spec.ts

@@ -0,0 +1,25 @@
+import { ComponentFixture, TestBed, waitForAsync } from '@angular/core/testing';
+
+import { EyeballerComponent } from './eyeballer.component';
+
+describe('EyeballerComponent', () => {
+  let component: EyeballerComponent;
+  let fixture: ComponentFixture<EyeballerComponent>;
+
+  beforeEach(waitForAsync(() => {
+    TestBed.configureTestingModule({
+      declarations: [ EyeballerComponent ]
+    })
+    .compileComponents();
+  }));
+
+  beforeEach(() => {
+    fixture = TestBed.createComponent(EyeballerComponent);
+    component = fixture.componentInstance;
+    fixture.detectChanges();
+  });
+
+  it('should create', () => {
+    expect(component).toBeTruthy();
+  });
+});

webui/src/app/components/eyeballer/eyeballer.component.ts

@@ -0,0 +1,264 @@
+import { Component, OnInit } from '@angular/core';
+import * as base64 from 'base64-arraybuffer';
+import * as tf from '@tensorflow/tfjs';
+
+
+@Component({
+  selector: 'app-eyeballer',
+  templateUrl: './eyeballer.component.html',
+  styleUrls: ['./eyeballer.component.scss']
+})
+export class EyeballerComponent implements OnInit {
+
+  offset = tf.scalar(127.5);
+  confidence = 0.5;
+
+  width = 224;
+  height = 224;
+
+  tfFilesCompleted = false;
+  tfFiles: File[] = [];
+  model: any = null;
+
+  imageCount = 0;
+  finishedLoading = false;
+  eyeballing = false;
+  eyeballCompleted = false;
+  eyeballedCount = 0;
+  classifications = {
+    custom404: [],
+    loginPage: [],
+    webapp: [],
+    oldLooking: [],
+    parked: [],
+  };
+
+  // What labels are selected?
+  // This is a tri-state:
+  //  0 means "always select"
+  //  1 means "don't care"
+  //  2 means "never select"
+  selected = {
+    custom404: 2,
+    loginPage: 1,
+    webapp: 0,
+    oldLooking: 1,
+    parked: 2,
+  }
+
+  selectedScreens: string[] = [];
+
+  constructor() {}
+
+  ngOnInit() {
+    this.fetchTfFiles();
+  }
+
+  async onSelect(event) {
+    this.imageCount = event.addedFiles.length;
+    this.eyeballing = true;
+    await this.classifyImages(event.addedFiles);
+    this.eyeballing = false;
+    this.eyeballCompleted = true;
+    await this.updateSelections();
+  }
+
+  async classifyImages(files) {
+    this.model = await tf.loadLayersModel(tf.io.browserFiles(this.tfFiles));
+    for(const file of files) {
+      await this.classify(file);
+      this.eyeballedCount++;
+    }
+  }
+
+  async classify(file) {
+    const dataString = await this.dataURI(file);
+    const img = new Image(this.width, this.height);
+    img.src = dataString;
+    img.onerror = () => {
+      return;
+    }
+    img.onload = () => {
+      tf.tidy(() => {
+        const tensor = tf.browser.fromPixels(img)
+          .resizeNearestNeighbor([224, 224])
+          .toFloat()
+          .sub(this.offset)
+          .div(this.offset)
+          .expandDims();
+        const resultTensor = <tf.Tensor<tf.Rank>> this.model.predict(tensor);
+        const predictions = resultTensor.dataSync();
+        tensor.dispose();
+        resultTensor.dispose();
+        if (predictions[0] > this.confidence) {
+          this.classifications.custom404.push(file.name);
+        }
+        if (predictions[1] > this.confidence) {
+          this.classifications.loginPage.push(file.name);
+        }
+        if (predictions[2] > this.confidence) {
+          this.classifications.webapp.push(file.name);
+        }
+        if (predictions[3] > this.confidence) {
+          this.classifications.oldLooking.push(file.name);
+        }
+        if (predictions[4] > this.confidence) {
+          this.classifications.parked.push(file.name);
+        }
+      });
+    };
+
+  }
+
+
+
+  isWebapp(key) {
+    if(this.classifications.webapp.includes(key)) {
+      return "Webapp"
+    }
+    return ""
+  }
+  isCustom404(key) {
+    if(this.classifications.custom404.includes(key)) {
+      return "Custom 404"
+    }
+    return ""
+  }
+  isLoginPage(key) {
+    if(this.classifications.loginPage.includes(key)) {
+      return "Login Page"
+    }
+    return ""
+  }
+  isOldLooking(key) {
+    if(this.classifications.oldLooking.includes(key)) {
+      return "Old Looking"
+    }
+    return ""
+  }
+  isParked(key) {
+    if(this.classifications.parked.includes(key)) {
+      return "Parked Domain"
+    }
+    return ""
+  }
+
+  async fetchTfFiles() {
+    let resp = await fetch('/assets/tf/model.json');
+    const manifest = await resp.json();
+    const paths: string[] = Array.from(manifest.weightsManifest[0]?.paths);
+
+    this.tfFiles = [];
+    resp = await fetch('/assets/tf/model.json');
+    const blob = await resp.blob();
+    this.tfFiles.push(new File([blob], 'model.json'));
+    await Promise.all(paths.map(async (path) => {
+      const tfFile = await this.fetchTfFile(path);
+      this.tfFiles.push(tfFile);
+    }));
+    this.tfFilesCompleted = true;
+  }
+
+  async fetchTfFile(name: string): Promise<File> {
+    const base = name.split('/').reverse()[0];
+    const resp = await fetch(`/assets/tf/${base}`);
+    const blob = await resp.blob();
+    return new File([blob], base);
+  }
+
+  async updateSelections() {
+    // First thing: load up all the images into the set
+    let selectedScreensSet = new Set([]);
+    for (let entry of this.classifications.webapp) {
+      selectedScreensSet.add(entry);
+    }
+    for (let entry of this.classifications.oldLooking) {
+      selectedScreensSet.add(entry);
+    }
+    for (let entry of this.classifications.loginPage) {
+      selectedScreensSet.add(entry);
+    }
+    for (let entry of this.classifications.custom404) {
+      selectedScreensSet.add(entry);
+    }
+    for (let entry of this.classifications.parked) {
+      selectedScreensSet.add(entry);
+    }
+    let selectionsArray = Array.from(selectedScreensSet);
+
+    // Calculate intersections for each "must include" selections
+    if (this.selected.webapp === 0){
+      let tempArray = selectionsArray.filter(value => this.classifications.webapp.includes(value));
+      selectionsArray = tempArray;
+    }
+    if (this.selected.oldLooking === 0){
+      let tempArray = selectionsArray.filter(value => this.classifications.oldLooking.includes(value));
+      selectionsArray = tempArray;
+    }
+    if (this.selected.loginPage === 0){
+      let tempArray = selectionsArray.filter(value => this.classifications.loginPage.includes(value));
+      selectionsArray = tempArray;
+    }
+    if (this.selected.custom404 === 0){
+      let tempArray = selectionsArray.filter(value => this.classifications.custom404.includes(value));
+      selectionsArray = tempArray;
+    }
+    if (this.selected.parked === 0){
+      let tempArray = selectionsArray.filter(value => this.classifications.parked.includes(value));
+      selectionsArray = tempArray;
+    }
+
+    // Now remove any "Must not include" selection
+    selectedScreensSet = new Set(selectionsArray);
+    if (this.selected.webapp === 2){
+      for (let entry of this.classifications.webapp) {
+        selectedScreensSet.delete(entry);
+      }
+    }
+    if (this.selected.oldLooking === 2){
+      for (let entry of this.classifications.oldLooking) {
+        selectedScreensSet.delete(entry);
+      }
+    }
+    if (this.selected.loginPage === 2){
+      for (let entry of this.classifications.loginPage) {
+        selectedScreensSet.delete(entry);
+      }
+    }
+    if (this.selected.custom404 === 2){
+      for (let entry of this.classifications.custom404) {
+        selectedScreensSet.delete(entry);
+      }
+    }
+    if (this.selected.parked === 2){
+      for (let entry of this.classifications.parked) {
+        selectedScreensSet.delete(entry);
+      }
+    }
+    this.selectedScreens = Array.from(selectedScreensSet);
+  }
+
+  async dataURI(file: File): Promise<string> {
+    const buf = await file.arrayBuffer();
+    let ext = file.name.split('.').reverse()[0]?.toLocaleLowerCase();
+    if (!["jpg", "jpeg", "png", "gif", "bmp"].some(allow => allow === ext)) {
+      ext = "jpg";
+    }
+    return `data:image/${encodeURIComponent(ext)};base64,${base64.encode(buf)}`;
+  }
+
+  eyeballPercent() {
+    return (this.eyeballedCount / this.imageCount) * 100;
+  }
+
+  restart() {
+    window.location.reload();
+  }
+
+  async exportResults() {
+    const str = this.selectedScreens.join("\n");
+    const blob = new Blob([str], { type: 'text/csv' });
+    const url= window.URL.createObjectURL(blob);
+    window.open(url);
+  }
+}

webui/src/assets/tf/model.json

@@ -0,0 +1 @@
+{"format": "layers-model", "generatedBy": "keras v2.2.4-tf", "convertedBy": "TensorFlow.js Converter v3.1.0", "modelTopology": {"keras_version": "2.2.4-tf", "backend": "tensorflow", "model_config": {"class_name": "Sequential", "config": {"name": "sequential", "layers": [{"class_name": "Model", "config": {"name": "mobilenet_1.00_224", "layers": [{"name": "input_1", "class_name": "InputLayer", "config": {"batch_input_shape": [null, 224, 224, 3], "dtype": "float32", "sparse": false, "ragged": false, "name": "input_1"}, "inbound_nodes": []}, {"name": "conv1_pad", "class_name": "ZeroPadding2D", "config": {"name": "conv1_pad", "trainable": true, "dtype": "float32", "padding": [[0, 1], [0, 1]], "data_format": "channels_last"}, "inbound_nodes": [[["input_1", 0, 0, {}]]]}, {"name": "conv1", "class_name": "Conv2D", "config": {"name": "conv1", "trainable": true, "dtype": "float32", "filters": 32, "kernel_size": [3, 3], "strides": [2, 2], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "inbound_nodes": [[["conv1_pad", 0, 0, {}]]]}, {"name": "conv1_bn", "class_name": "BatchNormalization", "config": {"name": "conv1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "moving_variance_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "inbound_nodes": [[["conv1", 0, 0, {}]]]}, {"name": "conv1_relu", "class_name": "ReLU", "config": {"name": "conv1_relu", "trainable": true, "dtype": "float32", "max_value": 6.0, "negative_slope": 0.0, "threshold": 0.0}, "inbound_nodes": [[["conv1_bn", 0, 0, {}]]]}, {"name": "conv_dw_1", "class_name": "DepthwiseConv2D", "config": {"name": "conv_dw_1", "trainable": true, "dtype": "float32", "kernel_size": [3, 3], "strides": [1, 1], "padding": "same", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "bias_regularizer": null, "activity_regularizer": null, "bias_constraint": null, "depth_multiplier": 1, "depthwise_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "depthwise_regularizer": null, "depthwise_constraint": null}, "inbound_nodes": [[["conv1_relu", 0, 0, {}]]]}, {"name": "conv_dw_1_bn", "class_name": "BatchNormalization", "config": {"name": "conv_dw_1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "moving_variance_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "inbound_nodes": [[["conv_dw_1", 0, 0, {}]]]}, {"name": "conv_dw_1_relu", "class_name": "ReLU", "config": {"name": "conv_dw_1_relu", "trainable": true, "dtype": "float32", "max_value": 6.0, "negative_slope": 0.0, "threshold": 0.0}, "inbound_nodes": [[["conv_dw_1_bn", 0, 0, {}]]]}, {"name": "conv_pw_1", "class_name": "Conv2D", "config": {"name": "conv_pw_1", "trainable": true, "dtype": "float32", "filters": 64, "kernel_size": [1, 1], "strides": [1, 1], "padding": "same", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "inbound_nodes": [[["conv_dw_1_relu", 0, 0, {}]]]}, {"name": "conv_pw_1_bn", "class_name": "BatchNormalization", "config": {"name": "conv_pw_1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_Line truncated

webui/src/environments/environment.prod.ts

@@ -0,0 +1,3 @@
+export const environment = {
+  production: true
+};

webui/src/environments/environment.ts

@@ -0,0 +1,16 @@
+// This file can be replaced during build by using the `fileReplacements` array.
+// `ng build --prod` replaces `environment.ts` with `environment.prod.ts`.
+// The list of file replacements can be found in `angular.json`.
+
+export const environment = {
+  production: false
+};
+
+/*
+ * For easier debugging in development mode, you can import the following file
+ * to ignore zone related error stack frames such as `zone.run`, `zoneDelegate.invokeTask`.
+ *
+ * This import should be commented out in production mode because it will have a negative impact
+ * on performance if an error is thrown.
+ */
+// import 'zone.js/dist/zone-error';  // Included with Angular CLI.

webui/src/index.html

@@ -0,0 +1,19 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <title>Eyeballer</title>
+  <base href="/">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <link rel="icon" type="image/x-icon" href="favicon.ico">
+</head>
+<body>
+  <app-root>
+    <h1>
+      <div style="text-align: center; margin: auto; width: 50%; padding: 50px 0;">
+        <i class="fas fa-spinner fa-spin fa-2x"></i>
+      </div>
+    </h1>
+  </app-root>
+</body>
+</html>

webui/src/main.ts

@@ -0,0 +1,12 @@
+import { enableProdMode } from '@angular/core';
+import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';
+
+import { AppModule } from './app/app.module';
+import { environment } from './environments/environment';
+
+if (environment.production) {
+  enableProdMode();
+}
+
+platformBrowserDynamic().bootstrapModule(AppModule)
+  .catch(err => console.error(err));

webui/src/polyfills.ts

@@ -0,0 +1,63 @@
+/**
+ * This file includes polyfills needed by Angular and is loaded before the app.
+ * You can add your own extra polyfills to this file.
+ *
+ * This file is divided into 2 sections:
+ *   1. Browser polyfills. These are applied before loading ZoneJS and are sorted by browsers.
+ *   2. Application imports. Files imported after ZoneJS that should be loaded before your main
+ *      file.
+ *
+ * The current setup is for so-called "evergreen" browsers; the last versions of browsers that
+ * automatically update themselves. This includes Safari >= 10, Chrome >= 55 (including Opera),
+ * Edge >= 13 on the desktop, and iOS 10 and Chrome on mobile.
+ *
+ * Learn more in https://angular.io/guide/browser-support
+ */
+
+/***************************************************************************************************
+ * BROWSER POLYFILLS
+ */
+
+/** IE10 and IE11 requires the following for NgClass support on SVG elements */
+// import 'classlist.js';  // Run `npm install --save classlist.js`.
+
+/**
+ * Web Animations `@angular/platform-browser/animations`
+ * Only required if AnimationBuilder is used within the application and using IE/Edge or Safari.
+ * Standard animation support in Angular DOES NOT require any polyfills (as of Angular 6.0).
+ */
+// import 'web-animations-js';  // Run `npm install --save web-animations-js`.
+
+/**
+ * By default, zone.js will patch all possible macroTask and DomEvents
+ * user can disable parts of macroTask/DomEvents patch by setting following flags
+ * because those flags need to be set before `zone.js` being loaded, and webpack
+ * will put import in the top of bundle, so user need to create a separate file
+ * in this directory (for example: zone-flags.ts), and put the following flags
+ * into that file, and then add the following code before importing zone.js.
+ * import './zone-flags';
+ *
+ * The flags allowed in zone-flags.ts are listed here.
+ *
+ * The following flags will work for all browsers.
+ *
+ * (window as any).__Zone_disable_requestAnimationFrame = true; // disable patch requestAnimationFrame
+ * (window as any).__Zone_disable_on_property = true; // disable patch onProperty such as onclick
+ * (window as any).__zone_symbol__UNPATCHED_EVENTS = ['scroll', 'mousemove']; // disable patch specified eventNames
+ *
+ *  in IE/Edge developer tools, the addEventListener will also be wrapped by zone.js
+ *  with the following flag, it will bypass `zone.js` patch for IE/Edge
+ *
+ *  (window as any).__Zone_enable_cross_context_check = true;
+ *
+ */
+
+/***************************************************************************************************
+ * Zone JS is required by default for Angular itself.
+ */
+import 'zone.js/dist/zone';  // Included with Angular CLI.
+
+
+/***************************************************************************************************
+ * APPLICATION IMPORTS
+ */

webui/src/styles.scss

@@ -0,0 +1,13 @@
+@import '@angular/material/prebuilt-themes/deeppurple-amber.css';
+
+html, body {
+  margin: 0;
+  padding: 0;
+  font-family: Roboto, Arial, sans-serif;
+
+  height: 90%;
+}
+
+body {
+  margin-top: 64px;
+}

webui/src/test.ts

@@ -0,0 +1,25 @@
+// This file is required by karma.conf.js and loads recursively all the .spec and framework files
+
+import 'zone.js/dist/zone-testing';
+import { getTestBed } from '@angular/core/testing';
+import {
+  BrowserDynamicTestingModule,
+  platformBrowserDynamicTesting
+} from '@angular/platform-browser-dynamic/testing';
+
+declare const require: {
+  context(path: string, deep?: boolean, filter?: RegExp): {
+    keys(): string[];
+    <T>(id: string): T;
+  };
+};
+
+// First, initialize the Angular testing environment.
+getTestBed().initTestEnvironment(
+  BrowserDynamicTestingModule,
+  platformBrowserDynamicTesting()
+);
+// Then we find all the tests.
+const context = require.context('./', true, /\.spec\.ts$/);
+// And load the modules.
+context.keys().map(context);

webui/tsconfig.app.json

@@ -0,0 +1,15 @@
+/* To learn more about this file see: https://angular.io/config/tsconfig. */
+{
+  "extends": "./tsconfig.json",
+  "compilerOptions": {
+    "outDir": "./out-tsc/app",
+    "types": []
+  },
+  "files": [
+    "src/main.ts",
+    "src/polyfills.ts"
+  ],
+  "include": [
+    "src/**/*.d.ts"
+  ]
+}

webui/tsconfig.json

@@ -0,0 +1,20 @@
+/* To learn more about this file see: https://angular.io/config/tsconfig. */
+{
+  "compileOnSave": false,
+  "compilerOptions": {
+    "baseUrl": "./",
+    "outDir": "./dist/out-tsc",
+    "sourceMap": true,
+    "declaration": false,
+    "downlevelIteration": true,
+    "experimentalDecorators": true,
+    "moduleResolution": "node",
+    "importHelpers": true,
+    "target": "es2015",
+    "module": "es2020",
+    "lib": [
+      "es2018",
+      "dom"
+    ]
+  }
+}

webui/tsconfig.spec.json

@@ -0,0 +1,18 @@
+/* To learn more about this file see: https://angular.io/config/tsconfig. */
+{
+  "extends": "./tsconfig.json",
+  "compilerOptions": {
+    "outDir": "./out-tsc/spec",
+    "types": [
+      "jasmine"
+    ]
+  },
+  "files": [
+    "src/test.ts",
+    "src/polyfills.ts"
+  ],
+  "include": [
+    "src/**/*.spec.ts",
+    "src/**/*.d.ts"
+  ]
+}

webui/tslint.json

@@ -0,0 +1,152 @@
+{
+  "extends": "tslint:recommended",
+  "rules": {
+    "align": {
+      "options": [
+        "parameters",
+        "statements"
+      ]
+    },
+    "array-type": false,
+    "arrow-return-shorthand": true,
+    "curly": true,
+    "deprecation": {
+      "severity": "warning"
+    },
+    "component-class-suffix": true,
+    "contextual-lifecycle": true,
+    "directive-class-suffix": true,
+    "directive-selector": [
+      true,
+      "attribute",
+      "app",
+      "camelCase"
+    ],
+    "component-selector": [
+      true,
+      "element",
+      "app",
+      "kebab-case"
+    ],
+    "eofline": true,
+    "import-blacklist": [
+      true,
+      "rxjs/Rx"
+    ],
+    "import-spacing": true,
+    "indent": {
+      "options": [
+        "spaces"
+      ]
+    },
+    "max-classes-per-file": false,
+    "max-line-length": [
+      true,
+      140
+    ],
+    "member-ordering": [
+      true,
+      {
+        "order": [
+          "static-field",
+          "instance-field",
+          "static-method",
+          "instance-method"
+        ]
+      }
+    ],
+    "no-console": [
+      true,
+      "debug",
+      "info",
+      "time",
+      "timeEnd",
+      "trace"
+    ],
+    "no-empty": false,
+    "no-inferrable-types": [
+      true,
+      "ignore-params"
+    ],
+    "no-non-null-assertion": true,
+    "no-redundant-jsdoc": true,
+    "no-switch-case-fall-through": true,
+    "no-var-requires": false,
+    "object-literal-key-quotes": [
+      true,
+      "as-needed"
+    ],
+    "quotemark": [
+      true,
+      "single"
+    ],
+    "semicolon": {
+      "options": [
+        "always"
+      ]
+    },
+    "space-before-function-paren": {
+      "options": {
+        "anonymous": "never",
+        "asyncArrow": "always",
+        "constructor": "never",
+        "method": "never",
+        "named": "never"
+      }
+    },
+    "typedef": [
+      true,
+      "call-signature"
+    ],
+    "typedef-whitespace": {
+      "options": [
+        {
+          "call-signature": "nospace",
+          "index-signature": "nospace",
+          "parameter": "nospace",
+          "property-declaration": "nospace",
+          "variable-declaration": "nospace"
+        },
+        {
+          "call-signature": "onespace",
+          "index-signature": "onespace",
+          "parameter": "onespace",
+          "property-declaration": "onespace",
+          "variable-declaration": "onespace"
+        }
+      ]
+    },
+    "variable-name": {
+      "options": [
+        "ban-keywords",
+        "check-format",
+        "allow-pascal-case"
+      ]
+    },
+    "whitespace": {
+      "options": [
+        "check-branch",
+        "check-decl",
+        "check-operator",
+        "check-separator",
+        "check-type",
+        "check-typecast"
+      ]
+    },
+    "no-conflicting-lifecycle": true,
+    "no-host-metadata-property": true,
+    "no-input-rename": true,
+    "no-inputs-metadata-property": true,
+    "no-output-native": true,
+    "no-output-on-prefix": true,
+    "no-output-rename": true,
+    "no-outputs-metadata-property": true,
+    "template-banana-in-box": true,
+    "template-no-negated-async": true,
+    "use-lifecycle-interface": true,
+    "use-pipe-transform-interface": true
+  },
+  "rulesDirectory": [
+    "codelyzer"
+  ]
+}