Compare commits

..
40 Commits
Author SHA1 Message Date
Dan Petro 921a8f061d Make asset paths relative 2021-06-07 10:09:39 -07:00
Dan Petro e94635ac00 Remove unused variable 2021-06-07 09:43:37 -07:00
Dan Petro 3c4dc31321 Scale usage to many thousands of images, disables previews 2021-06-04 17:02:50 -07:00
Dan Petro 5f28534fae Fix webgl memory leaks
- Apparently, tfjs doesn't do memory management for you!
2021-05-18 13:49:37 -07:00
Dan Petro 83523d1a23 Batch classification calls 2021-05-18 11:45:02 -07:00
Dan Petro 2d7d338893 Confidence threshold set to 0.5 2021-05-14 08:30:01 -07:00
Dan Petro e28e6381f1 Handle error when non-image file is loaded 2021-05-13 10:18:38 -07:00
Dan Petro ac8414b486 Remove debug logging 2021-05-13 09:58:11 -07:00
Dan Petro 4c65c677f4 Remove start button. Begin eyeballing right away when files dropped 2021-05-12 13:37:25 -07:00
Dan Petro ade4b37f2b More minor UI changes 2021-05-11 16:04:34 -07:00
Dan Petro 57bdd8c5ab Add source code footer and labels explanation link 2021-05-11 15:13:09 -07:00
Dan Petro 08df5f4604 Disable start button until some files are uploaded 2021-05-11 14:17:56 -07:00
Dan Petro 25da9bcc2b Some minor UI improvements 2021-05-11 14:08:18 -07:00
Dan Petro d191b3284d Better styling 2021-05-10 16:45:26 -07:00
Dan Petro aaa1d206ba Initial angular webui code 2021-05-10 10:25:46 -07:00
Dan Petro 79b6f9f808 Fix typo in readme 2021-04-27 11:38:43 -07:00
Dan Petro 19cbae58e5 Added performance stats to readme 2021-04-27 11:32:53 -07:00
Dan Petro a390bf5644 Documentation for v3.0 training data 2021-04-22 14:01:04 -07:00
Dan Petro 92738b16a4 Add verify util script 2021-04-21 16:23:30 -07:00
Dan Petro 361b47bea8 Change resizer script to properly resize to 224x224, not 224x140 2021-04-21 08:22:57 -07:00
Dan Petro 616116d9a2 Documentation for parked domains 2021-04-20 15:28:52 -07:00
Dan Petro ecef89c4f9 Add extra field in prebuilt html template for parked domains 2021-04-20 14:46:29 -07:00
Dan Petro 818166f607 Support parked domains in the code 2021-04-02 13:56:46 -07:00
Dan Petro c50eae4dc4 Update readme with pretrained weights instructions 2021-03-01 16:49:40 -07:00
Dan Petro 3c8e161f82 Updated readme to say correct pretrained network name 2021-03-01 13:59:46 -07:00
Dan Petro 3b39a0c244 Merge branch 'application-label' 2021-03-01 13:57:53 -07:00
Dan Petro 42a4d6d037 Change readme dropbox link to v2 images/labels/pretrained model 2021-03-01 13:53:33 -07:00
Dan Petro c8eec3de4a Change name in HTML output file to "webapp" 2021-02-22 13:03:37 -07:00
Dan Petro 28234dac76 Trying logo again 2021-02-18 16:16:00 -07:00
Dan Petro bb46582e74 Minor logo readme fix 2021-02-18 16:15:07 -07:00
Dan Petro 33086f2f78 Add logo to readme 2021-02-18 16:13:46 -07:00
Dan Petro a3be157768 Change name from homepage to webapp 2021-02-18 14:51:16 -07:00
Dan Petro 8703982fe0 Readme clarification on label meanings 2021-02-12 11:51:34 -07:00
Dan Petro c90d90c07c Removed some commented code 2021-02-11 11:33:10 -07:00
Dan Petro 87e7ce3741 Fix bug where predict woulnd't load model from file
- Update some minor API things to not use deprecated features
2021-02-11 11:13:27 -07:00
Dan Petro ba6d6f0f49 Remove deprecated fit_generator() in tf2.0 2021-02-05 16:12:17 -07:00
Dan Petro 99fd136a0d Moved label init so that it's not necessary for prediction
- IE: So you don't need a labels.csv file if you just want to use a pretrained model
2021-02-04 15:50:20 -07:00
Dan Petro b7063bc517 De-lint, gitignore 2021-02-04 15:18:54 -07:00
Pi Esposito b031377f83 Added tensorboard callback for later visualization 2021-02-04 14:30:59 -07:00
Dan Petro b6e1f630f5 TF 2.0 is default now, so remove warning from readme 2021-02-04 14:27:27 -07:00
57 changed files with 35504 additions and 320 deletions

No files matched your search

+4 -7
View File
@@ -104,10 +104,9 @@ venv.bak/
.mypy_cache/
#screenshot data
images*
#Graphs and Results
graphs/*
images/*.png
*.jpg
*.tar
#model files
*.h5
@@ -120,6 +119,4 @@ results.html
!tests/models/*
!tests/data/*
!docs/*
#Leave the webui things alone for their own gitignore
!webui/*
logs/*
+57 -31
View File
@@ -1,4 +1,8 @@
# Eyeballer
![Logo](/docs/eyeballer_logo.png)
Give those screenshots of yours a quick eyeballing.
Eyeballer is meant for large-scope network penetration tests where you need to find "interesting" targets from a huge set of web-based hosts. Go ahead and use your favorite screenshotting tool like normal (EyeWitness or GoWitness) and then run them through Eyeballer to tell you what's likely to contain vulnerabilities, and what isn't.
@@ -10,38 +14,32 @@ Eyeballer is meant for large-scope network penetration tests where you need to f
| ------ |:-----:|
| ![Sample Old-looking Page](/docs/old-looking.png) | ![Sample Login Page](/docs/login.png) |
| Homepages | Custom 404's |
| Webapp | Custom 404's |
| ------ |:-----:|
| ![Sample Homepage](/docs/homepage.png) | ![Sample Custom 404](/docs/404.png) |
| ![Sample Webapp](/docs/homepage.png) | ![Sample Custom 404](/docs/404.png) |
## Web UI Setup
| Parked Domains |
| ------ |
| ![Sample Webapp](/docs/parked.png) |
Eyeballer's Web UI uses TensorflowJS and React. It runs entirely in your browser locally, with no backend functionality whatsoever. So you can "upload" files to classify and get results without the files ever leaving your computer. To run the Eyeballer Web UI locally, you'll need Node. On Debian, you can install that with the npm package:
## What the Labels Mean
```
sudo apt install npm
```
**Old-Looking Sites**
Blocky frames, broken CSS, that certain "je ne sais quoi" of a website that looks like it was designed in the early 2000's. You know it when you see it. Old websites aren't just ugly, they're also typically super vulnerable. When you're looking to hack into something, these websites are a gold mine.
Then you'll need to install some Node dependencies:
**Login Pages**
Login pages are valuable to pen testing, they indicate that there's additional functionality you don't currently have access to. It also means there's a simple follow-up process of credential enumeration attacks. You might think that you can set a simple heuristic to find login pages, but in practice it's really hard. Modern sites don't just use a simple input tag we can grep for.
```
cd webui/
npm install
```
**Webapp**
This tells you that there is a larger group of pages and functionality available here that can serve as surface area to attack. This is in contrast to a simple login page, with no other functionality. Or a default IIS landing page which has no other functionality. This label should indicate to you that there is a web application here to attack.
Copy a pre-trained model file (in TensorflowJS format) into a jsmodel/ folder under public/, so that it's available in the web server.
**Custom 404**
Modern sites love to have cutesy custom 404 pages with pictures of broken robots or sad looking dogs. Unfortunately, they also love to return HTTP 200 response codes while they do it. More often, the "404" page doesn't even contain the text "404" in it. These pages are typically uninteresting, despite having a lot going on visually, and Eyeballer can help you sift them out.
```
cp YOUR_MODEL/ webui/public/
```
**Parked Domains**
Parked domains are websites that look real, but aren't valid attack surface. They're stand-in pages, usually devoid of any real functionality, consist almost entirely of ads, and are usually not run by our actual target. It's what you get when the domain specified is wrong or lapsed. Finding these pages and removing them from scope is really valuable over time.
Then you can run the Web UI with:
```
npm start
```
## Python Setup
## Setup
Download required packages on pip:
```
@@ -55,33 +53,61 @@ sudo pip3 install -r requirements-gpu.txt
**NOTE**: Setting up a GPU for use with TensorFlow is way beyond the scope of this README. There's hardware compatibility to consider, drivers to install... There's a lot. So you're just going to have to figure this part out on your own if you want a GPU. But at least from a Python package perspective, the above requirements file has you covered.
**Pretrained Weights**
For the latest pretrained weights, check out the releases here on GitHub.
**Training Data** You can find our training data here:
https://www.dropbox.com/sh/7aouywaid7xptpq/AAD_-I4hAHrDeiosDAQksnBma?dl=1
https://www.dropbox.com/s/rpylhiv2g0kokts/eyeballer-3.0.zip?dl=1
Pretty soon, we're going to add this as a TensorFlow DataSet, so you don't need to download this separately like this. It'll also let us version the data a bit better. But for now, just deal with it. There's two things you need from the training data:
There's two things you need from the training data:
1. `images/` folder, containing all the screenshots (resized down to 224x140. We'll have the full-size images up soon)
1. `images/` folder, containing all the screenshots (resized down to 224x224)
2. `labels.csv` that has all the labels
3. `bishop-fox-pretrained-v1.h5` A pretrained weights file you can use right out of the box without training.
3. `bishop-fox-pretrained-v3.h5` A pretrained weights file you can use right out of the box without training.
Copy all three into the root of the Eyeballer code tree.
## Predicting Labels
NOTE: For best results, make sure you screenshot your websites in a native 1.6x aspect ratio. IE: 1440x900. Eyeballer will scale the image down automatically to the right size for you, but if it's the wrong aspect ratio then it will squish in a way that will affect prediction performance.
To eyeball some screenshots, just run the "predict" mode:
```
eyeballer.py --model YOUR_MODEL.h5 predict YOUR_FILE.png
eyeballer.py --weights YOUR_WEIGHTS.h5 predict YOUR_FILE.png
```
Or for a whole directory of files:
```
eyeballer.py --model YOUR_MODEL.h5 predict PATH_TO/YOUR_FILES/
eyeballer.py --weights YOUR_WEIGHTS.h5 predict PATH_TO/YOUR_FILES/
```
Eyeballer will spit the results back to you in human readable format (a `results.html` file so you can browse it easily) and machine readable format (a `results.csv` file).
## Performance
Eyeballer's performance is measured against an evaluation dataset, which is 20% of the overall screenshots chosen at random. Since these screenshots are never used in training, they can be an effective way to see how well the model is performing. Here are the latest results:
| Overall Binary Accuracy | 93.52% |
| ------ |:-----:|
| **All-or-Nothing Accuracy** | **76.09%** |
**Overall Binary Accuracy** is probably what you think of as the model's "accuracy". It's the chance, given any single label, that it is correct.
**All-or-Nothing Accuracy** is more strict. For this, we consider all of an image's labels and consider it a failure if ANY label is wrong. This accuracy rating is the chance that the model correctly predicts all labels for any given image.
| Label | Precision | Recall |
| ------ | ------ |:-----:|
| Custom 404 | 80.20% | 91.01% |
| Login Page | 86.41% | 88.47% |
| Webapp | 95.32% | 96.83% |
| Old Looking | 91.70% | 62.20% |
| Parked Domain | 70.99% | 66.43% |
For a detailed explanation on [Precision vs Recall, check out Wikipedia](https://en.wikipedia.org/wiki/Precision_and_recall).
## Training
To train a new model, run:
@@ -91,14 +117,14 @@ eyeballer.py train
You'll want a machine with a good GPU for this to run in a reasonable amount of time. Setting that up is outside the scope of this readme, however.
This will output a new model file (model.h5 by default).
This will output a new model file (weights.h5 by default).
## Evaluation
You just trained a new model, cool! Let's see how well it performs against some images it's never seen before, across a variety of metrics:
```
eyeballer.py --model YOUR_MODEL.h5 evaluate
eyeballer.py --weights YOUR_WEIGHTS.h5 evaluate
```
The output will describe the model's accuracy in both recall and precision for each of the program's labels. (Including "none of the above" as a pseudo-label)
Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

+4 -4
View File
@@ -9,12 +9,12 @@ from jinja2 import Template
@click.group(invoke_without_command=True)
@click.option('--model', default=None, type=click.Path(), help="Model file for input/output")
@click.option('--weights', default=None, type=click.Path(), help="Weights file for input/output")
@click.option('--summary/--no-summary', default=False, help="Print model summary at start")
@click.option('--seed', default=None, type=int, help="RNG seed for data shuffling and transformations, defaults to random value")
@click.pass_context
def cli(ctx, model, summary, seed):
model_kwargs = {"model_file": model,
def cli(ctx, weights, summary, seed):
model_kwargs = {"weights_file": weights,
"print_summary": summary,
"seed": seed}
@@ -54,7 +54,7 @@ def predict(ctx, screenshot, heatmap, threshold):
print(results)
else:
with open("results.csv", "w", newline="") as csvfile:
fieldnames = ["filename", "custom404", "login", "homepage", "oldlooking"]
fieldnames = ["filename", "custom404", "login", "webapp", "oldlooking", "parked"]
labelwriter = csv.DictWriter(csvfile, fieldnames=fieldnames)
labelwriter.writeheader()
labelwriter.writerows(results)
+45 -34
View File
@@ -17,7 +17,7 @@ from tensorflow.keras.applications.mobilenet import preprocess_input
from sklearn.metrics import classification_report, accuracy_score, hamming_loss
from eyeballer.augmentation import EyeballerAugmentation
DATA_LABELS = ["custom404", "login", "homepage", "oldlooking"]
DATA_LABELS = ["custom404", "login", "webapp", "oldlooking", "parked"]
class EyeballModel:
@@ -26,17 +26,17 @@ class EyeballModel:
Contains high-level functions for training, evaluating, and predicting.
"""
graphs_directory = "graphs/"
checkpoint_file = "model.h5"
checkpoint_file = "weights.h5"
image_dir = "images/"
image_width, image_height = 224, 224
input_shape = (image_width, image_height, 3)
def __init__(self, model_file, print_summary=False, seed=None, quiet=False):
def __init__(self, weights_file, print_summary=False, seed=None, quiet=False):
"""Constructor for model class.
Keyword arguments:
print_summary -- Whether or not to print to stdout the keras model summary, containing a detailed description of every model layer
model_file -- A filename for where to load the model's model.
weights_file -- A filename for where to load the model's weights.
seed -- PRNG seed, useful for repeating a previous run and using the same data. Training/Validation split is determined randomly.
"""
# # Build the model
@@ -54,14 +54,25 @@ class EyeballModel:
loss="binary_crossentropy",
metrics=["accuracy"])
# for layer in pretrained_layer.layers:
# layer.trainable = False
if weights_file is not None and os.path.isfile(weights_file):
try:
self.model.load_weights(weights_file)
except OSError:
print("ERROR: Unable to open weights file '{}'".foramt(weights_file))
sys.exit(-1)
print("Loaded model from file.")
else:
if weights_file is not None:
raise FileNotFoundError
print("WARN: No model loaded from file. Generating random model")
if print_summary:
print(self.model.summary())
self.quiet = quiet
self.seed = seed
def _init_labels(self):
# Pull out our labels for use in generators later
data = pd.read_csv("labels.csv")
self.training_labels = data.loc[data['evaluation'] == False] # noqa: E712
@@ -69,7 +80,6 @@ class EyeballModel:
# Shuffle the training labels
self.random_seed = False
self.seed = seed
if self.seed is None:
self.random_seed = True
self.seed = random.randint(0, 999999)
@@ -78,18 +88,6 @@ class EyeballModel:
random.seed(self.seed)
self.training_labels = self.training_labels.sample(frac=1)
if model_file is not None and os.path.isfile(model_file):
try:
self.model = tf.keras.models.load_model(model_file)
except OSError:
print("ERROR: Unable to open model file '{}'".format(model_file))
sys.exit(-1)
print("Loaded model from file.")
else:
if model_file is not None:
raise FileNotFoundError
print("WARN: No model loaded from file. Generating random model")
# Data augmentation
augmentor = Augmentor.Pipeline()
augmentor.set_seed(self.seed)
@@ -105,7 +103,7 @@ class EyeballModel:
self.preprocess_training_function = augmentor.keras_preprocess_func()
def train(self, epochs=20, batch_size=32, print_graphs=False):
"""Train the model, making a new model file at each successfull checkpoint. You'll probably need a GPU for this to realistically run.
"""Train the model, making a new weights file at each successfull checkpoint. You'll probably need a GPU for this to realistically run.
Keyword arguments:
epochs -- The number of epochs to train for. (An epoch is one pass-through of the dataset)
@@ -114,6 +112,7 @@ class EyeballModel:
"""
print("Training with seed: " + str(self.seed))
self._init_labels()
data_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=self.preprocess_training_function,
validation_split=0.2,
@@ -129,7 +128,7 @@ class EyeballModel:
subset='training',
shuffle=True,
seed=self.seed,
class_mode="other")
class_mode="raw")
validation_generator = data_generator.flow_from_dataframe(
self.training_labels,
directory=self.image_dir,
@@ -140,17 +139,26 @@ class EyeballModel:
subset='validation',
shuffle=False,
seed=self.seed,
class_mode="other")
class_mode="raw")
# Model checkpoint - Saves model when validation accuracy improves
# Model checkpoint - Saves model weights when validation accuracy improves
callbacks = [tf.keras.callbacks.ModelCheckpoint(self.checkpoint_file,
monitor='val_loss',
verbose=1,
save_best_only=True,
save_weights_only=False,
mode='min')]
monitor='val_loss',
verbose=1,
save_best_only=True,
save_weights_only=True,
mode='min'),
tf.keras.callbacks.TensorBoard(log_dir='logs',
histogram_freq=2,
write_graph=True,
write_images=False,
update_freq='epoch',
profile_batch=2,
embeddings_freq=0,
embeddings_metadata=None),
]
history = self.model.fit_generator(
history = self.model.fit(
training_generator,
steps_per_epoch=len(training_generator.filenames) // batch_size,
epochs=epochs,
@@ -197,8 +205,9 @@ class EyeballModel:
result["filename"] = "custom-image"
result["custom404"] = prediction[0][0]
result["login"] = prediction[0][1]
result["homepage"] = prediction[0][2]
result["webapp"] = prediction[0][2]
result["oldlooking"] = prediction[0][3]
result["parked"] = prediction[0][4]
return result
def predict(self, path, threshold=0.5):
@@ -241,8 +250,9 @@ class EyeballModel:
result["filename"] = screenshot
result["custom404"] = prediction[0][0]
result["login"] = prediction[0][1]
result["homepage"] = prediction[0][2]
result["webapp"] = prediction[0][2]
result["oldlooking"] = prediction[0][3]
result["parked"] = prediction[0][4]
results.append(result)
return results
@@ -253,6 +263,7 @@ class EyeballModel:
threshold -- Value between 0->1. The cutoff where the numerical prediction becomes boolean. Default: 0.5
"""
# Prepare the data
self._init_labels()
data_generator = tf.keras.preprocessing.image.ImageDataGenerator(
preprocessing_function=preprocess_input)
evaluation_generator = data_generator.flow_from_dataframe(
@@ -263,7 +274,7 @@ class EyeballModel:
target_size=(self.image_width, self.image_height),
shuffle=False,
batch_size=1,
class_mode="other")
class_mode="raw")
# If a seed was selected, then also evaluate on the validation set for that seed
if not self.random_seed:
print("Using validation set...")
@@ -282,11 +293,11 @@ class EyeballModel:
subset='validation',
shuffle=False,
seed=self.seed,
class_mode="other")
class_mode="raw")
else:
print("Using evaluation set...")
predictions = self.model.predict_generator(
predictions = self.model.predict(
evaluation_generator,
verbose=1,
steps=len(evaluation_generator))
+2 -1
View File
@@ -7,8 +7,9 @@
<button class="btn all" onclick="filterSelection('all')"> Show all</button>
<button class="btn" onclick="filterSelection('custom404')"> 404</button>
<button class="btn" onclick="filterSelection('login')"> Login</button>
<button class="btn" onclick="filterSelection('homepage')"> Home Page</button>
<button class="btn" onclick="filterSelection('webapp')"> Web Application</button>
<button class="btn" onclick="filterSelection('oldlooking')"> Old Looking</button>
<button class="btn" onclick="filterSelection('parked')"> Parked Domain</button>
<select onchange="numOnPage=this.value; filterSelection([]);" style="float:right; margin:5px; margin-right:30px; height:30px; width:auto;">
<option value="25">25</option>
<option value="50">50</option>
+7 -7
View File
@@ -15,16 +15,16 @@ class PredictTest(unittest.TestCase):
def flush(self): pass
sys.stdout = DummyFile()
model_file = "tests/models/model_file.h5"
if not os.path.isfile(model_file):
print("Error: Symlink the latest model file to " + model_file)
weights_file = "tests/models/test_weights.h5"
if not os.path.isfile(weights_file):
print("Error: Symlink the latest weights file to " + weights_file)
raise FileNotFoundError(
errno.ENOENT,
os.strerror(errno.ENOENT),
model_file)
weights_file)
model_kwargs = {
"model_file": model_file,
"weights_file": weights_file,
"print_summary": False,
"seed": None,
"quiet": True
@@ -33,7 +33,7 @@ class PredictTest(unittest.TestCase):
def test_different_seed_predict(self):
model_kwargs = {
"model_file": None,
"weights_file": None,
"print_summary": False,
"seed": 12345678,
"quiet": True
@@ -49,7 +49,7 @@ class PredictTest(unittest.TestCase):
def test_same_seed_predict(self):
model_kwargs = {
"model_file": None,
"weights_file": None,
"print_summary": False,
"seed": 12345678,
"quiet": True
+3 -2
View File
@@ -1,7 +1,8 @@
#!/bin/bash
# Converts full-size images to a given size (replacing the originals)
for f in images_full_size/*
for f in images/*
do
convert -resize 224x224 $f $f
echo $f
convert -resize 224x224\! $f $f
done
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env python3
import csv
import random
from os import listdir
from os.path import isfile, join
with open("labels.csv", newline="") as csvfile:
# Get the header labels
csvreader = csv.DictReader(csvfile)
# fieldnames = next(csvreader)
file_list = [f for f in listdir("images") if isfile(join("images", f))]
label_list = []
for row in csvreader:
label_list.append(row["filename"])
# Loop through the file
print("Rows in labels.csv, but no image exists:")
for label in label_list:
if label not in file_list:
print(row["filename"])
print("Images that exist, but don't have labels:")
i = 0
for image in file_list:
if image not in label_list:
print(image)
+40 -12
View File
@@ -1,20 +1,48 @@
# See https://help.github.com/ignore-files/ for more about ignoring files.
# See http://help.github.com/ignore-files/ for more about ignoring files.
# compiled output
/dist
/tmp
/out-tsc
# Only exists if Bazel was run
/bazel-out
.vscode/
# dependencies
/node_modules
# testing
/coverage
# profiling files
chrome-profiler-events*.json
speed-measure-plugin*.json
# production
/build
# IDEs and editors
/.idea
.project
.classpath
.c9/
*.launch
.settings/
*.sublime-workspace
# IDE - VSCode
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
.history/*
# misc
.DS_Store
.env
npm-debug.log*
yarn-debug.log*
yarn-error.log*
/.sass-cache
/connect.lock
/coverage
/libpeerconnection.log
npm-debug.log
yarn-error.log
testem.log
/typings
#Model files
public/jsmodel/*
# System Files
.DS_Store
Thumbs.db
+11
View File
@@ -0,0 +1,11 @@
# Weballer
Web front-end for Eyeballer
### Setup
```
npm install
ng serve
```
+133
View File
@@ -0,0 +1,133 @@
{
"$schema": "./node_modules/@angular/cli/lib/config/schema.json",
"version": 1,
"newProjectRoot": "projects",
"projects": {
"weballer": {
"projectType": "application",
"schematics": {
"@schematics/angular:component": {
"style": "scss"
}
},
"root": "",
"sourceRoot": "src",
"prefix": "app",
"architect": {
"build": {
"builder": "@angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist/weballer",
"index": "src/index.html",
"main": "src/main.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "tsconfig.app.json",
"aot": true,
"assets": [
"src/favicon.ico",
"src/assets"
],
"styles": [
"./node_modules/@angular/material/prebuilt-themes/deeppurple-amber.css",
"./node_modules/@fortawesome/fontawesome-free/css/all.min.css",
"src/styles.scss"
],
"scripts": []
},
"configurations": {
"production": {
"fileReplacements": [
{
"replace": "src/environments/environment.ts",
"with": "src/environments/environment.prod.ts"
}
],
"optimization": true,
"outputHashing": "all",
"sourceMap": false,
"namedChunks": false,
"extractLicenses": true,
"vendorChunk": false,
"buildOptimizer": true,
"budgets": [
{
"type": "initial",
"maximumWarning": "2mb",
"maximumError": "5mb"
},
{
"type": "anyComponentStyle",
"maximumWarning": "6kb",
"maximumError": "10kb"
}
]
}
}
},
"serve": {
"builder": "@angular-devkit/build-angular:dev-server",
"options": {
"browserTarget": "weballer:build"
},
"configurations": {
"production": {
"browserTarget": "weballer:build:production"
}
}
},
"extract-i18n": {
"builder": "@angular-devkit/build-angular:extract-i18n",
"options": {
"browserTarget": "weballer:build"
}
},
"test": {
"builder": "@angular-devkit/build-angular:karma",
"options": {
"main": "src/test.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "tsconfig.spec.json",
"karmaConfig": "karma.conf.js",
"assets": [
"src/favicon.ico",
"src/assets"
],
"styles": [
"src/styles.scss"
],
"scripts": []
}
},
"lint": {
"builder": "@angular-devkit/build-angular:tslint",
"options": {
"tsConfig": [
"tsconfig.app.json",
"tsconfig.spec.json",
"e2e/tsconfig.json"
],
"exclude": [
"**/node_modules/**"
]
}
},
"e2e": {
"builder": "@angular-devkit/build-angular:protractor",
"options": {
"protractorConfig": "e2e/protractor.conf.js",
"devServerTarget": "weballer:serve"
},
"configurations": {
"production": {
"devServerTarget": "weballer:serve:production"
}
}
}
}
}
},
"defaultProject": "weballer",
"cli": {
"analytics": false
}
}
+36
View File
@@ -0,0 +1,36 @@
// @ts-check
// Protractor configuration file, see link for more information
// https://github.com/angular/protractor/blob/master/lib/config.ts
const { SpecReporter, StacktraceOption } = require('jasmine-spec-reporter');
/**
* @type { import("protractor").Config }
*/
exports.config = {
allScriptsTimeout: 11000,
specs: [
'./src/**/*.e2e-spec.ts'
],
capabilities: {
browserName: 'chrome'
},
directConnect: true,
baseUrl: 'http://localhost:4200/',
framework: 'jasmine',
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 30000,
print: function() {}
},
onPrepare() {
require('ts-node').register({
project: require('path').join(__dirname, './tsconfig.json')
});
jasmine.getEnv().addReporter(new SpecReporter({
spec: {
displayStacktrace: StacktraceOption.PRETTY
}
}));
}
};
+23
View File
@@ -0,0 +1,23 @@
import { AppPage } from './app.po';
import { browser, logging } from 'protractor';
describe('workspace-project App', () => {
let page: AppPage;
beforeEach(() => {
page = new AppPage();
});
it('should display welcome message', () => {
page.navigateTo();
expect(page.getTitleText()).toEqual('weballer app is running!');
});
afterEach(async () => {
// Assert that there are no errors emitted from the browser
const logs = await browser.manage().logs().get(logging.Type.BROWSER);
expect(logs).not.toContain(jasmine.objectContaining({
level: logging.Level.SEVERE,
} as logging.Entry));
});
});
+11
View File
@@ -0,0 +1,11 @@
import { browser, by, element } from 'protractor';
export class AppPage {
navigateTo(): Promise<unknown> {
return browser.get(browser.baseUrl) as Promise<unknown>;
}
getTitleText(): Promise<string> {
return element(by.css('app-root .content span')).getText() as Promise<string>;
}
}
+14
View File
@@ -0,0 +1,14 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"extends": "../tsconfig.json",
"compilerOptions": {
"outDir": "../out-tsc/e2e",
"module": "commonjs",
"target": "es2018",
"types": [
"jasmine",
"jasminewd2",
"node"
]
}
}
+32
View File
@@ -0,0 +1,32 @@
// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine', '@angular-devkit/build-angular'],
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular-devkit/build-angular/plugins/karma')
],
client: {
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
coverageIstanbulReporter: {
dir: require('path').join(__dirname, './coverage/weballer'),
reports: ['html', 'lcovonly', 'text-summary'],
fixWebpackSourcePaths: true
},
reporters: ['progress', 'kjhtml'],
port: 9876,
colors: true,
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['Chrome'],
singleRun: false,
restartOnFileChange: true
});
};
+33973
View File
File diff suppressed because it is too large. Load diff
+48 -14
View File
@@ -1,20 +1,54 @@
{
"name": "eyeballer-webui",
"version": "0.1.0",
"name": "weballer",
"version": "0.0.0",
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build",
"test": "ng test",
"lint": "ng lint",
"e2e": "ng e2e"
},
"private": true,
"dependencies": {
"react": "^16.11.0",
"react-dom": "^16.11.0",
"react-dropzone": "^10.1.10",
"react-scripts": "0.9.5",
"bootstrap": "^4.3.1",
"@tensorflow/tfjs": "^1.3.2"
"@angular/animations": "~11.2.6",
"@angular/cdk": "^11.2.5",
"@angular/common": "~11.2.6",
"@angular/compiler": "~11.2.6",
"@angular/core": "~11.2.6",
"@angular/flex-layout": "^11.0.0-beta.33",
"@angular/forms": "~11.2.6",
"@angular/material": "^11.2.5",
"@angular/platform-browser": "~11.2.6",
"@angular/platform-browser-dynamic": "~11.2.6",
"@angular/router": "~11.2.6",
"@fortawesome/fontawesome-free": "^5.15.3",
"@tensorflow/tfjs": "^3.3.0",
"base64-arraybuffer": "^0.2.0",
"ngx-dropzone": "^3.0.0",
"rxjs": "~6.6.6",
"tslib": "^2.0.0",
"zone.js": "~0.10.3"
},
"devDependencies": {},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test --env=jsdom",
"eject": "react-scripts eject"
"devDependencies": {
"@angular-devkit/build-angular": "^0.1102.5",
"@angular/cli": "~11.2.5",
"@angular/compiler-cli": "~11.2.6",
"@types/jasmine": "~3.6.0",
"@types/jasminewd2": "~2.0.3",
"@types/node": "^12.11.1",
"@types/webgl2": "0.0.6",
"codelyzer": "^6.0.0",
"jasmine-core": "~3.6.0",
"jasmine-spec-reporter": "~5.0.0",
"karma": "~6.2.0",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage-istanbul-reporter": "~3.0.2",
"karma-jasmine": "~4.0.0",
"karma-jasmine-html-reporter": "^1.5.0",
"protractor": "~7.0.0",
"ts-node": "~8.3.0",
"tslint": "~6.1.0",
"typescript": "~4.1.5"
}
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.6 KiB

-31
View File
@@ -1,31 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.png">
<!--
Notice the use of %PUBLIC_URL% in the tag above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>React App</title>
</head>
<body>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start`.
To create a production bundle, use `npm run build`.
-->
</body>
</html>
-23
View File
@@ -1,23 +0,0 @@
.App {
text-align: center;
}
.App-logo {
height: 100px;
}
.App-header {
background-color: #222;
height: 170px;
padding: 10px;
color: white;
}
.App-intro {
font-size: large;
}
.thumbnail {
height: 280px;
width: 480px;
}
-132
View File
@@ -1,132 +0,0 @@
import React, { Component } from 'react';
import logo from './eyeballer_logo.png';
import './App.css';
import 'bootstrap/dist/css/bootstrap.min.css';
import * as tf from '@tensorflow/tfjs';
class App extends Component {
constructor(props) {
super(props);
this.state = {
fileContents: "Try uploading a file",
}
}
render() {
return (
<div className="App">
<div className="App-header">
<img src={logo} className="App-logo" alt="logo" />
<h2>Eyeballer</h2>
</div>
<PredictButton />
<ImageFile />
</div>
);
}
}
export default App;
class ImageFile extends React.Component {
constructor(props){
super(props);
this.state = {
id : "someUniqueId", // I would use this.props.id for a real world implementation
imageURI : null
}
}
buildImgTag(){
let imgTag = null;
if (this.state.imageURI !== null)
imgTag = (<div className="row">
<div className="small-9 small-centered columns">
<img className="thumbnail" id="thumbnail" alt="Screenshot to be classified" src={this.state.imageURI}></img>
</div>
</div>);
return imgTag;
}
readURI(e){
if(e.target.files && e.target.files[0]){
let reader = new FileReader();
reader.onload = function(ev){
this.setState({imageURI:ev.target.result});
}.bind(this);
reader.readAsDataURL(e.target.files[0]);
}
}
handleChange(e){
this.readURI(e); // maybe call this with webworker or async library?
if (this.props.onChange !== undefined)
this.props.onChange(e); // propagate to parent component
}
render() {
const imgTag = this.buildImgTag();
return <div>
<label
htmlFor={this.state.id}
className="button">
Upload an image
</label>
<input
id={this.state.id}
type="file"
onChange={this.handleChange.bind(this)}
className="show-for-sr" />
{imgTag}
</div>;
}
}
class PredictButton extends Component {
constructor(props) {
super(props);
this.state = {
classification: ""
}
}
onpress = async () => {
const modelUrl = 'jsmodel/model.json';
const model = await tf.loadLayersModel(modelUrl);
const offset = tf.scalar(127.5);
const image = tf.browser.fromPixels(document.getElementById('thumbnail'))
.resizeNearestNeighbor([224,224])
.toFloat()
.sub(offset)
.div(offset)
.expandDims();
const predictions = Array.from(model.predict(image).dataSync());
this.setState({classification: ""});
if (predictions[0] > 0.5) {
this.setState({classification: this.state.classification + "Custom 404, "});
}
if (predictions[1] > 0.5) {
this.setState({classification: this.state.classification + "Login Page, "});
}
if (predictions[2] > 0.5) {
this.setState({classification: this.state.classification + "Homepage, "});
}
if (predictions[3] > 0.5) {
this.setState({classification: this.state.classification + "Old Looking "});
}
}
render() {
return (
<div className="custom-file">
<button className="default" type="button" id="predictButton" onClick={this.onpress}>Classify</button>
{this.state.classification}
</div>
);
}
}
-8
View File
@@ -1,8 +0,0 @@
import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';
it('renders without crashing', () => {
const div = document.createElement('div');
ReactDOM.render(<App />, div);
});
+13
View File
@@ -0,0 +1,13 @@
import { NgModule } from '@angular/core';
import { Routes, RouterModule } from '@angular/router';
import { EyeballerComponent } from './components/eyeballer/eyeballer.component';
const routes: Routes = [
{ path: '', component: EyeballerComponent },
];
@NgModule({
imports: [RouterModule.forRoot(routes, { relativeLinkResolution: 'legacy' })],
exports: [RouterModule]
})
export class AppRoutingModule { }
+2
View File
@@ -0,0 +1,2 @@
<router-outlet>
<router-outlet>
View File
Whitespace-only changes.
+35
View File
@@ -0,0 +1,35 @@
import { TestBed, waitForAsync } from '@angular/core/testing';
import { RouterTestingModule } from '@angular/router/testing';
import { AppComponent } from './app.component';
describe('AppComponent', () => {
beforeEach(waitForAsync(() => {
TestBed.configureTestingModule({
imports: [
RouterTestingModule
],
declarations: [
AppComponent
],
}).compileComponents();
}));
it('should create the app', () => {
const fixture = TestBed.createComponent(AppComponent);
const app = fixture.componentInstance;
expect(app).toBeTruthy();
});
it(`should have as title 'weballer'`, () => {
const fixture = TestBed.createComponent(AppComponent);
const app = fixture.componentInstance;
expect(app.title).toEqual('weballer');
});
it('should render title', () => {
const fixture = TestBed.createComponent(AppComponent);
fixture.detectChanges();
const compiled = fixture.nativeElement;
expect(compiled.querySelector('.content span').textContent).toContain('weballer app is running!');
});
});
+10
View File
@@ -0,0 +1,10 @@
import { Component } from '@angular/core';
@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.scss']
})
export class AppComponent {
title = 'weballer';
}
+28
View File
@@ -0,0 +1,28 @@
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FlexLayoutModule } from "@angular/flex-layout";
import { BrowserAnimationsModule } from '@angular/platform-browser/animations';
import { NgxDropzoneModule } from 'ngx-dropzone';
import { BaseMaterialModule } from './base-materials';
import { AppRoutingModule } from './app-routing.module';
import { AppComponent } from './app.component';
import { EyeballerComponent } from './components/eyeballer/eyeballer.component';
@NgModule({
declarations: [
AppComponent,
EyeballerComponent
],
imports: [
BrowserModule,
AppRoutingModule,
NgxDropzoneModule,
BrowserAnimationsModule,
BaseMaterialModule,
FlexLayoutModule
],
providers: [],
bootstrap: [AppComponent]
})
export class AppModule { }
+58
View File
@@ -0,0 +1,58 @@
/*
Electric Scan
Copyright (C) 2019 Bishop Fox
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
*/
import { NgModule } from '@angular/core';
import { MatBadgeModule } from '@angular/material/badge';
import { MatButtonModule } from '@angular/material/button';
import { MatCardModule } from '@angular/material/card';
import { MatCheckboxModule } from '@angular/material/checkbox';
import { MatOptionModule } from '@angular/material/core';
import { MatDialogModule } from '@angular/material/dialog';
import { MatDividerModule } from '@angular/material/divider';
import { MatFormFieldModule } from '@angular/material/form-field';
import { MatGridListModule } from '@angular/material/grid-list';
import { MatIconModule } from '@angular/material/icon';
import { MatInputModule } from '@angular/material/input';
import { MatMenuModule } from '@angular/material/menu';
import { MatProgressBarModule } from '@angular/material/progress-bar';
import { MatProgressSpinnerModule } from '@angular/material/progress-spinner';
import { MatSelectModule } from '@angular/material/select';
import { MatSidenavModule } from '@angular/material/sidenav';
import { MatSlideToggleModule } from '@angular/material/slide-toggle';
import { MatSnackBarModule } from '@angular/material/snack-bar';
import { MatSortModule } from '@angular/material/sort';
import { MatTableModule } from '@angular/material/table';
import { MatTabsModule } from '@angular/material/tabs';
import { MatToolbarModule } from '@angular/material/toolbar';
import { MatTooltipModule } from '@angular/material/tooltip';
import { MatExpansionModule } from '@angular/material/expansion';
const modules = [
MatButtonModule, MatCheckboxModule, MatOptionModule, MatSelectModule, MatMenuModule, MatTabsModule,
MatFormFieldModule, MatTooltipModule, MatToolbarModule, MatIconModule, MatProgressSpinnerModule,
MatDialogModule, MatGridListModule, MatCardModule, MatTableModule, MatSortModule, MatInputModule,
MatSnackBarModule, MatSlideToggleModule, MatDividerModule, MatProgressBarModule, MatBadgeModule,
MatSidenavModule, MatExpansionModule
];
@NgModule({
imports: modules,
exports: modules,
})
export class BaseMaterialModule { }
@@ -0,0 +1,114 @@
<mat-toolbar class="app-toolbar" color="primary" fxLayout="row" fxLayoutAlign="start center">
<h2>
<img src="assets/eyeballer_logo.png" width="46" height="60">
</h2>
Eyeballer
<span fxFlex></span>
<span>Bishop Fox R&D</span>
<span fxFlex></span>
<button mat-raised-button (click)="restart()">
<i class="fas fa-fw fa-undo"></i>
Restart
</button>
</mat-toolbar>
<!-- Model Loading Progress -->
<div *ngIf="!tfFilesCompleted">
<mat-progress-bar mode="indeterminate"></mat-progress-bar>
</div>
<!-- Drop Zone -->
<div *ngIf="tfFilesCompleted && !eyeballing && !eyeballCompleted">
<mat-card>
Drop your screenshots below:
<br><br>
<ngx-dropzone (change)="onSelect($event)">
<ngx-dropzone-label>
Files dropped here are <span style="font-weight: bold">not uploaded anywhere.</span> No data leaves your computer at all on this tool.
</ngx-dropzone-label>
</ngx-dropzone>
</mat-card>
<br />
<mat-card><span style="font-weight: bold">Note:</span> For best results,
ensure your screenshots are taken natively in a <span style="font-weight: bold">16:10 aspect ratio.</span> (ie: 1440x900 or other)
</mat-card>
<mat-card>
Source code available at <a href="https://github.com/BishopFox/eyeballer" target="_blank" rel="noopener noreferrer">GitHub</a>
</mat-card>
</div>
<!-- Progress Bar -->
<div *ngIf="tfFilesCompleted && eyeballing && !eyeballCompleted">
<mat-card>
<h3>Classifying: {{this.eyeballedCount}} of {{this.imageCount}}</h3>
<mat-progress-bar mode="determinate" [value]="eyeballPercent()"></mat-progress-bar>
</mat-card>
<mat-card>
This processing is happening in your browser. <span style="font-weight: bold">There is no backend.</span>
</mat-card>
</div>
<!-- Results -->
<div *ngIf="tfFilesCompleted && !eyeballing && eyeballCompleted">
<div class="results-container">
<h2>Results</h2>
<mat-toolbar>
<mat-toolbar-row style="height:40px;">
<span class="button-spacer"></span>
<button class="label-button" mat-raised-button color="primary" [disabled]="selected.webapp === 0" (click)="selected.webapp = 0; updateSelections()">Must Include</button>
<button class="label-button" mat-raised-button color="basic" [disabled]="selected.webapp === 1" (click)="selected.webapp = 1; updateSelections()">Webapp</button>
<button class="label-button" mat-raised-button color="accent" [disabled]="selected.webapp === 2" (click)="selected.webapp = 2; updateSelections()">Never Include</button>
<label class="label-count">{{ classifications.webapp.length }}</label>
<span class="button-spacer"></span>
</mat-toolbar-row>
<mat-toolbar-row style="height:40px;">
<span class="button-spacer"></span>
<button class="label-button" mat-raised-button color="primary" [disabled]="selected.oldLooking === 0" (click)="selected.oldLooking = 0; updateSelections()">Must Include</button>
<button class="label-button" mat-raised-button color="basic" [disabled]="selected.oldLooking === 1" (click)="selected.oldLooking = 1; updateSelections()">Old Looking</button>
<button class="label-button" mat-raised-button color="accent" [disabled]="selected.oldLooking === 2" (click)="selected.oldLooking = 2; updateSelections()">Never Include</button>
<label class="label-count">{{ classifications.oldLooking.length }}</label>
<span class="button-spacer"></span>
</mat-toolbar-row>
<mat-toolbar-row style="height:40px;">
<span class="button-spacer"></span>
<button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.loginPage === 0" (click)="selected.loginPage = 0; updateSelections()">Must Include</button>
<button class="label-button" mat-raised-button color="basic" [disabled]="selected.loginPage === 1" (click)="selected.loginPage = 1; updateSelections()">Login Page</button>
<button class="label-button" mat-raised-button color="accent" [disabled]="selected.loginPage === 2" (click)="selected.loginPage = 2; updateSelections()">Never Include</button>
<label class="label-count">{{ classifications.loginPage.length }}</label>
<span class="button-spacer"></span>
</mat-toolbar-row>
<mat-toolbar-row style="height:40px;">
<span class="button-spacer"></span>
<button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.custom404 === 0" (click)="selected.custom404 = 0; updateSelections()">Must Include</button>
<button class="label-button" mat-raised-button color="basic" [disabled]="selected.custom404 === 1" (click)="selected.custom404 = 1; updateSelections()">Custom 404</button>
<button class="label-button" mat-raised-button color="accent" [disabled]="selected.custom404 === 2" (click)="selected.custom404 = 2; updateSelections()">Never Include</button>
<label class="label-count">{{ classifications.custom404.length }}</label>
<span class="button-spacer"></span>
</mat-toolbar-row>
<mat-toolbar-row style="height:40px;">
<span class="button-spacer"></span>
<button class="label-button" mat-raised-button color="primary" mat-raised-button [disabled]="selected.parked === 0" (click)="selected.parked = 0; updateSelections()">Must Include</button>
<button class="label-button" mat-raised-button color="basic" [disabled]="selected.parked === 1" (click)="selected.parked = 1; updateSelections()">Parked Domain</button>
<button class="label-button" mat-raised-button color="accent" [disabled]="selected.parked === 2" (click)="selected.parked = 2; updateSelections()">Never Include</button>
<label class="label-count">{{ classifications.parked.length }}</label>
<span class="button-spacer"></span>
</mat-toolbar-row>
</mat-toolbar>
<mat-card>
<span style="font-weight: bold">Total Selected:</span> {{ selectedScreens.length }}
</mat-card>
<br>
<mat-card>
<button class="export-button" mat-raised-button color="primary" (click)="exportResults()">
Export
</button>
<a href="https://github.com/BishopFox/eyeballer#what-the-labels-mean" target="_blank" rel="noopener noreferrer">What do these labels mean?</a>
</mat-card>
</div>
</div>
@@ -0,0 +1,128 @@
.app-toolbar {
position: fixed;
position: -webkit-sticky; /* For macOS/iOS Safari */
top: 0; /* Sets the sticky toolbar to be on top */
z-index: 1000; /* Ensure that your app's content doesn't overlap the toolbar */
}
.not-completed-container {
display: grid;
justify-items: center;
align-items: center;
grid-template-columns: 1fr 1fr 1fr;
}
.not-completed-message {
grid-column: 2/3;
grid-row: 2/3;
}
.results-container {
display: grid;
justify-items: center;
align-items: center;
grid-template-columns: 1fr;
}
.eyeballed-class-container {
display: grid;
justify-items: center;
align-items: center;
// grid-template-rows: minmax(250px, auto);
grid-template-columns: 1fr 1fr 1fr 1fr;
grid-gap: 1em;
margin: 1em;
}
.result-card {
max-width: 100%;
}
.result-card:hover {
background-color: #f5f5f5;
}
.result-card:hover img {
opacity: .5;
}
.result-card > div {
display: none;
position: absolute;
top: 50%;
left: 50%;
width: 70px;
height: 70px;
z-index: 10;
}
.fill-remaining-space {
flex: 1 1 auto;
}
.fill-space {
flex: auto;
}
.img-container {
position: relative;
text-align: center;
color: white;
}
.preview-image {
width: 100%;
}
.webapp-label {
background-color: blue;
color: white;
margin-left: 1px;
margin-right: 1px;
}
.old-looking-label {
background-color: green;
color: white;
margin-left: 1px;
margin-right: 1px;
}
.login-label {
background-color: purple;
color: white;
margin-left: 1px;
margin-right: 1px;
}
.custom-404-label {
background-color: orange;
color: white;
margin-left: 1px;
margin-right: 1px;
}
.parked-label {
background-color: red;
color: white;
margin-left: 1px;
margin-right: 1px;
}
.label-count {
margin-left: 10px;
width: 50px!important;
min-width: unset!important;
}
.label-button{
width: 130px!important;
min-width: unset!important;
}
.button-spacer {
flex: 1 1 auto;
}
.export-button {
margin-right: 10px;
}
@@ -0,0 +1,25 @@
import { ComponentFixture, TestBed, waitForAsync } from '@angular/core/testing';
import { EyeballerComponent } from './eyeballer.component';
describe('EyeballerComponent', () => {
let component: EyeballerComponent;
let fixture: ComponentFixture<EyeballerComponent>;
beforeEach(waitForAsync(() => {
TestBed.configureTestingModule({
declarations: [ EyeballerComponent ]
})
.compileComponents();
}));
beforeEach(() => {
fixture = TestBed.createComponent(EyeballerComponent);
component = fixture.componentInstance;
fixture.detectChanges();
});
it('should create', () => {
expect(component).toBeTruthy();
});
});
@@ -0,0 +1,263 @@
import { Component, OnInit } from '@angular/core';
import * as base64 from 'base64-arraybuffer';
import * as tf from '@tensorflow/tfjs';
@Component({
selector: 'app-eyeballer',
templateUrl: './eyeballer.component.html',
styleUrls: ['./eyeballer.component.scss']
})
export class EyeballerComponent implements OnInit {
offset = tf.scalar(127.5);
confidence = 0.5;
width = 224;
height = 224;
tfFilesCompleted = false;
tfFiles: File[] = [];
model: any = null;
imageCount = 0;
eyeballing = false;
eyeballCompleted = false;
eyeballedCount = 0;
classifications = {
custom404: [],
loginPage: [],
webapp: [],
oldLooking: [],
parked: [],
};
// What labels are selected?
// This is a tri-state:
// 0 means "always select"
// 1 means "don't care"
// 2 means "never select"
selected = {
custom404: 2,
loginPage: 1,
webapp: 0,
oldLooking: 1,
parked: 2,
}
selectedScreens: string[] = [];
constructor() {}
ngOnInit() {
this.fetchTfFiles();
}
async onSelect(event) {
this.imageCount = event.addedFiles.length;
this.eyeballing = true;
await this.classifyImages(event.addedFiles);
this.eyeballing = false;
this.eyeballCompleted = true;
await this.updateSelections();
}
async classifyImages(files) {
this.model = await tf.loadLayersModel(tf.io.browserFiles(this.tfFiles));
for(const file of files) {
await this.classify(file);
this.eyeballedCount++;
}
}
async classify(file) {
const dataString = await this.dataURI(file);
const img = new Image(this.width, this.height);
img.src = dataString;
img.onerror = () => {
return;
}
img.onload = () => {
tf.tidy(() => {
const tensor = tf.browser.fromPixels(img)
.resizeNearestNeighbor([224, 224])
.toFloat()
.sub(this.offset)
.div(this.offset)
.expandDims();
const resultTensor = <tf.Tensor<tf.Rank>> this.model.predict(tensor);
const predictions = resultTensor.dataSync();
tensor.dispose();
resultTensor.dispose();
if (predictions[0] > this.confidence) {
this.classifications.custom404.push(file.name);
}
if (predictions[1] > this.confidence) {
this.classifications.loginPage.push(file.name);
}
if (predictions[2] > this.confidence) {
this.classifications.webapp.push(file.name);
}
if (predictions[3] > this.confidence) {
this.classifications.oldLooking.push(file.name);
}
if (predictions[4] > this.confidence) {
this.classifications.parked.push(file.name);
}
});
};
}
isWebapp(key) {
if(this.classifications.webapp.includes(key)) {
return "Webapp"
}
return ""
}
isCustom404(key) {
if(this.classifications.custom404.includes(key)) {
return "Custom 404"
}
return ""
}
isLoginPage(key) {
if(this.classifications.loginPage.includes(key)) {
return "Login Page"
}
return ""
}
isOldLooking(key) {
if(this.classifications.oldLooking.includes(key)) {
return "Old Looking"
}
return ""
}
isParked(key) {
if(this.classifications.parked.includes(key)) {
return "Parked Domain"
}
return ""
}
async fetchTfFiles() {
let resp = await fetch('assets/tf/model.json');
const manifest = await resp.json();
const paths: string[] = Array.from(manifest.weightsManifest[0]?.paths);
this.tfFiles = [];
resp = await fetch('assets/tf/model.json');
const blob = await resp.blob();
this.tfFiles.push(new File([blob], 'model.json'));
await Promise.all(paths.map(async (path) => {
const tfFile = await this.fetchTfFile(path);
this.tfFiles.push(tfFile);
}));
this.tfFilesCompleted = true;
}
async fetchTfFile(name: string): Promise<File> {
const base = name.split('/').reverse()[0];
const resp = await fetch(`assets/tf/${base}`);
const blob = await resp.blob();
return new File([blob], base);
}
async updateSelections() {
// First thing: load up all the images into the set
let selectedScreensSet = new Set([]);
for (let entry of this.classifications.webapp) {
selectedScreensSet.add(entry);
}
for (let entry of this.classifications.oldLooking) {
selectedScreensSet.add(entry);
}
for (let entry of this.classifications.loginPage) {
selectedScreensSet.add(entry);
}
for (let entry of this.classifications.custom404) {
selectedScreensSet.add(entry);
}
for (let entry of this.classifications.parked) {
selectedScreensSet.add(entry);
}
let selectionsArray = Array.from(selectedScreensSet);
// Calculate intersections for each "must include" selections
if (this.selected.webapp === 0){
let tempArray = selectionsArray.filter(value => this.classifications.webapp.includes(value));
selectionsArray = tempArray;
}
if (this.selected.oldLooking === 0){
let tempArray = selectionsArray.filter(value => this.classifications.oldLooking.includes(value));
selectionsArray = tempArray;
}
if (this.selected.loginPage === 0){
let tempArray = selectionsArray.filter(value => this.classifications.loginPage.includes(value));
selectionsArray = tempArray;
}
if (this.selected.custom404 === 0){
let tempArray = selectionsArray.filter(value => this.classifications.custom404.includes(value));
selectionsArray = tempArray;
}
if (this.selected.parked === 0){
let tempArray = selectionsArray.filter(value => this.classifications.parked.includes(value));
selectionsArray = tempArray;
}
// Now remove any "Must not include" selection
selectedScreensSet = new Set(selectionsArray);
if (this.selected.webapp === 2){
for (let entry of this.classifications.webapp) {
selectedScreensSet.delete(entry);
}
}
if (this.selected.oldLooking === 2){
for (let entry of this.classifications.oldLooking) {
selectedScreensSet.delete(entry);
}
}
if (this.selected.loginPage === 2){
for (let entry of this.classifications.loginPage) {
selectedScreensSet.delete(entry);
}
}
if (this.selected.custom404 === 2){
for (let entry of this.classifications.custom404) {
selectedScreensSet.delete(entry);
}
}
if (this.selected.parked === 2){
for (let entry of this.classifications.parked) {
selectedScreensSet.delete(entry);
}
}
this.selectedScreens = Array.from(selectedScreensSet);
}
async dataURI(file: File): Promise<string> {
const buf = await file.arrayBuffer();
let ext = file.name.split('.').reverse()[0]?.toLocaleLowerCase();
if (!["jpg", "jpeg", "png", "gif", "bmp"].some(allow => allow === ext)) {
ext = "jpg";
}
return `data:image/${encodeURIComponent(ext)};base64,${base64.encode(buf)}`;
}
eyeballPercent() {
return (this.eyeballedCount / this.imageCount) * 100;
}
restart() {
window.location.reload();
}
async exportResults() {
const str = this.selectedScreens.join("\n");
const blob = new Blob([str], { type: 'text/csv' });
const url= window.URL.createObjectURL(blob);
window.open(url);
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+1
View File
@@ -0,0 +1 @@
{"format": "layers-model", "generatedBy": "keras v2.2.4-tf", "convertedBy": "TensorFlow.js Converter v3.1.0", "modelTopology": {"keras_version": "2.2.4-tf", "backend": "tensorflow", "model_config": {"class_name": "Sequential", "config": {"name": "sequential", "layers": [{"class_name": "Model", "config": {"name": "mobilenet_1.00_224", "layers": [{"name": "input_1", "class_name": "InputLayer", "config": {"batch_input_shape": [null, 224, 224, 3], "dtype": "float32", "sparse": false, "ragged": false, "name": "input_1"}, "inbound_nodes": []}, {"name": "conv1_pad", "class_name": "ZeroPadding2D", "config": {"name": "conv1_pad", "trainable": true, "dtype": "float32", "padding": [[0, 1], [0, 1]], "data_format": "channels_last"}, "inbound_nodes": [[["input_1", 0, 0, {}]]]}, {"name": "conv1", "class_name": "Conv2D", "config": {"name": "conv1", "trainable": true, "dtype": "float32", "filters": 32, "kernel_size": [3, 3], "strides": [2, 2], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "inbound_nodes": [[["conv1_pad", 0, 0, {}]]]}, {"name": "conv1_bn", "class_name": "BatchNormalization", "config": {"name": "conv1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "moving_variance_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "inbound_nodes": [[["conv1", 0, 0, {}]]]}, {"name": "conv1_relu", "class_name": "ReLU", "config": {"name": "conv1_relu", "trainable": true, "dtype": "float32", "max_value": 6.0, "negative_slope": 0.0, "threshold": 0.0}, "inbound_nodes": [[["conv1_bn", 0, 0, {}]]]}, {"name": "conv_dw_1", "class_name": "DepthwiseConv2D", "config": {"name": "conv_dw_1", "trainable": true, "dtype": "float32", "kernel_size": [3, 3], "strides": [1, 1], "padding": "same", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "bias_regularizer": null, "activity_regularizer": null, "bias_constraint": null, "depth_multiplier": 1, "depthwise_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "depthwise_regularizer": null, "depthwise_constraint": null}, "inbound_nodes": [[["conv1_relu", 0, 0, {}]]]}, {"name": "conv_dw_1_bn", "class_name": "BatchNormalization", "config": {"name": "conv_dw_1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "moving_variance_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "beta_regularizer": null, "gamma_regularizer": null, "beta_constraint": null, "gamma_constraint": null}, "inbound_nodes": [[["conv_dw_1", 0, 0, {}]]]}, {"name": "conv_dw_1_relu", "class_name": "ReLU", "config": {"name": "conv_dw_1_relu", "trainable": true, "dtype": "float32", "max_value": 6.0, "negative_slope": 0.0, "threshold": 0.0}, "inbound_nodes": [[["conv_dw_1_bn", 0, 0, {}]]]}, {"name": "conv_pw_1", "class_name": "Conv2D", "config": {"name": "conv_pw_1", "trainable": true, "dtype": "float32", "filters": 64, "kernel_size": [1, 1], "strides": [1, 1], "padding": "same", "data_format": "channels_last", "dilation_rate": [1, 1], "activation": "linear", "use_bias": false, "kernel_initializer": {"class_name": "GlorotUniform", "config": {"seed": null, "dtype": "float32"}}, "bias_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}, "inbound_nodes": [[["conv_dw_1_relu", 0, 0, {}]]]}, {"name": "conv_pw_1_bn", "class_name": "BatchNormalization", "config": {"name": "conv_pw_1_bn", "trainable": true, "dtype": "float32", "axis": [3], "momentum": 0.99, "epsilon": 0.001, "center": true, "scale": true, "beta_initializer": {"class_name": "Zeros", "config": {"dtype": "float32"}}, "gamma_initializer": {"class_name": "Ones", "config": {"dtype": "float32"}}, "moving_mean_initializer": {"class_Line truncated
@@ -0,0 +1,3 @@
export const environment = {
production: true
};
+16
View File
@@ -0,0 +1,16 @@
// This file can be replaced during build by using the `fileReplacements` array.
// `ng build --prod` replaces `environment.ts` with `environment.prod.ts`.
// The list of file replacements can be found in `angular.json`.
export const environment = {
production: false
};
/*
* For easier debugging in development mode, you can import the following file
* to ignore zone related error stack frames such as `zone.run`, `zoneDelegate.invokeTask`.
*
* This import should be commented out in production mode because it will have a negative impact
* on performance if an error is thrown.
*/
// import 'zone.js/dist/zone-error'; // Included with Angular CLI.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 962 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 948 B

-5
View File
@@ -1,5 +0,0 @@
body {
margin: 0;
padding: 0;
font-family: sans-serif;
}
+19
View File
@@ -0,0 +1,19 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Eyeballer</title>
<base href=".">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/x-icon" href="favicon.ico">
</head>
<body>
<app-root>
<h1>
<div style="text-align: center; margin: auto; width: 50%; padding: 50px 0;">
<i class="fas fa-spinner fa-spin fa-2x"></i>
</div>
</h1>
</app-root>
</body>
</html>
-9
View File
@@ -1,9 +0,0 @@
import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';
import './index.css';
ReactDOM.render(
<App />,
document.getElementById('root')
);
+12
View File
@@ -0,0 +1,12 @@
import { enableProdMode } from '@angular/core';
import { platformBrowserDynamic } from '@angular/platform-browser-dynamic';
import { AppModule } from './app/app.module';
import { environment } from './environments/environment';
if (environment.production) {
enableProdMode();
}
platformBrowserDynamic().bootstrapModule(AppModule)
.catch(err => console.error(err));
+63
View File
@@ -0,0 +1,63 @@
/**
* This file includes polyfills needed by Angular and is loaded before the app.
* You can add your own extra polyfills to this file.
*
* This file is divided into 2 sections:
* 1. Browser polyfills. These are applied before loading ZoneJS and are sorted by browsers.
* 2. Application imports. Files imported after ZoneJS that should be loaded before your main
* file.
*
* The current setup is for so-called "evergreen" browsers; the last versions of browsers that
* automatically update themselves. This includes Safari >= 10, Chrome >= 55 (including Opera),
* Edge >= 13 on the desktop, and iOS 10 and Chrome on mobile.
*
* Learn more in https://angular.io/guide/browser-support
*/
/***************************************************************************************************
* BROWSER POLYFILLS
*/
/** IE10 and IE11 requires the following for NgClass support on SVG elements */
// import 'classlist.js'; // Run `npm install --save classlist.js`.
/**
* Web Animations `@angular/platform-browser/animations`
* Only required if AnimationBuilder is used within the application and using IE/Edge or Safari.
* Standard animation support in Angular DOES NOT require any polyfills (as of Angular 6.0).
*/
// import 'web-animations-js'; // Run `npm install --save web-animations-js`.
/**
* By default, zone.js will patch all possible macroTask and DomEvents
* user can disable parts of macroTask/DomEvents patch by setting following flags
* because those flags need to be set before `zone.js` being loaded, and webpack
* will put import in the top of bundle, so user need to create a separate file
* in this directory (for example: zone-flags.ts), and put the following flags
* into that file, and then add the following code before importing zone.js.
* import './zone-flags';
*
* The flags allowed in zone-flags.ts are listed here.
*
* The following flags will work for all browsers.
*
* (window as any).__Zone_disable_requestAnimationFrame = true; // disable patch requestAnimationFrame
* (window as any).__Zone_disable_on_property = true; // disable patch onProperty such as onclick
* (window as any).__zone_symbol__UNPATCHED_EVENTS = ['scroll', 'mousemove']; // disable patch specified eventNames
*
* in IE/Edge developer tools, the addEventListener will also be wrapped by zone.js
* with the following flag, it will bypass `zone.js` patch for IE/Edge
*
* (window as any).__Zone_enable_cross_context_check = true;
*
*/
/***************************************************************************************************
* Zone JS is required by default for Angular itself.
*/
import 'zone.js/dist/zone'; // Included with Angular CLI.
/***************************************************************************************************
* APPLICATION IMPORTS
*/
+13
View File
@@ -0,0 +1,13 @@
@import '@angular/material/prebuilt-themes/deeppurple-amber.css';
html, body {
margin: 0;
padding: 0;
font-family: Roboto, Arial, sans-serif;
height: 90%;
}
body {
margin-top: 64px;
}
+25
View File
@@ -0,0 +1,25 @@
// This file is required by karma.conf.js and loads recursively all the .spec and framework files
import 'zone.js/dist/zone-testing';
import { getTestBed } from '@angular/core/testing';
import {
BrowserDynamicTestingModule,
platformBrowserDynamicTesting
} from '@angular/platform-browser-dynamic/testing';
declare const require: {
context(path: string, deep?: boolean, filter?: RegExp): {
keys(): string[];
<T>(id: string): T;
};
};
// First, initialize the Angular testing environment.
getTestBed().initTestEnvironment(
BrowserDynamicTestingModule,
platformBrowserDynamicTesting()
);
// Then we find all the tests.
const context = require.context('./', true, /\.spec\.ts$/);
// And load the modules.
context.keys().map(context);
+15
View File
@@ -0,0 +1,15 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "./out-tsc/app",
"types": []
},
"files": [
"src/main.ts",
"src/polyfills.ts"
],
"include": [
"src/**/*.d.ts"
]
}
+20
View File
@@ -0,0 +1,20 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"compileOnSave": false,
"compilerOptions": {
"baseUrl": "./",
"outDir": "./dist/out-tsc",
"sourceMap": true,
"declaration": false,
"downlevelIteration": true,
"experimentalDecorators": true,
"moduleResolution": "node",
"importHelpers": true,
"target": "es2015",
"module": "es2020",
"lib": [
"es2018",
"dom"
]
}
}
+18
View File
@@ -0,0 +1,18 @@
/* To learn more about this file see: https://angular.io/config/tsconfig. */
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "./out-tsc/spec",
"types": [
"jasmine"
]
},
"files": [
"src/test.ts",
"src/polyfills.ts"
],
"include": [
"src/**/*.spec.ts",
"src/**/*.d.ts"
]
}
+152
View File
@@ -0,0 +1,152 @@
{
"extends": "tslint:recommended",
"rules": {
"align": {
"options": [
"parameters",
"statements"
]
},
"array-type": false,
"arrow-return-shorthand": true,
"curly": true,
"deprecation": {
"severity": "warning"
},
"component-class-suffix": true,
"contextual-lifecycle": true,
"directive-class-suffix": true,
"directive-selector": [
true,
"attribute",
"app",
"camelCase"
],
"component-selector": [
true,
"element",
"app",
"kebab-case"
],
"eofline": true,
"import-blacklist": [
true,
"rxjs/Rx"
],
"import-spacing": true,
"indent": {
"options": [
"spaces"
]
},
"max-classes-per-file": false,
"max-line-length": [
true,
140
],
"member-ordering": [
true,
{
"order": [
"static-field",
"instance-field",
"static-method",
"instance-method"
]
}
],
"no-console": [
true,
"debug",
"info",
"time",
"timeEnd",
"trace"
],
"no-empty": false,
"no-inferrable-types": [
true,
"ignore-params"
],
"no-non-null-assertion": true,
"no-redundant-jsdoc": true,
"no-switch-case-fall-through": true,
"no-var-requires": false,
"object-literal-key-quotes": [
true,
"as-needed"
],
"quotemark": [
true,
"single"
],
"semicolon": {
"options": [
"always"
]
},
"space-before-function-paren": {
"options": {
"anonymous": "never",
"asyncArrow": "always",
"constructor": "never",
"method": "never",
"named": "never"
}
},
"typedef": [
true,
"call-signature"
],
"typedef-whitespace": {
"options": [
{
"call-signature": "nospace",
"index-signature": "nospace",
"parameter": "nospace",
"property-declaration": "nospace",
"variable-declaration": "nospace"
},
{
"call-signature": "onespace",
"index-signature": "onespace",
"parameter": "onespace",
"property-declaration": "onespace",
"variable-declaration": "onespace"
}
]
},
"variable-name": {
"options": [
"ban-keywords",
"check-format",
"allow-pascal-case"
]
},
"whitespace": {
"options": [
"check-branch",
"check-decl",
"check-operator",
"check-separator",
"check-type",
"check-typecast"
]
},
"no-conflicting-lifecycle": true,
"no-host-metadata-property": true,
"no-input-rename": true,
"no-inputs-metadata-property": true,
"no-output-native": true,
"no-output-on-prefix": true,
"no-output-rename": true,
"no-outputs-metadata-property": true,
"template-banana-in-box": true,
"template-no-negated-async": true,
"use-lifecycle-interface": true,
"use-pipe-transform-interface": true
},
"rulesDirectory": [
"codelyzer"
]
}