Compare commits

...
40 Commits
Author SHA1 Message Date
moloch-- 868b2c294c Merge branch 'master' into fix/dnsc2 2025-12-30 23:04:32 -06:00
moloch-- 9827b4df4b Add forms pkg 2025-12-30 23:04:03 -06:00
moloch-- a48309a813 Add forms pkg 2025-12-30 23:03:55 -06:00
moloch-- 5df5dec5d4 Add forms pkg 2025-12-30 23:03:46 -06:00
Joe 3ff3ef2ed4 Merge pull request #2076 from BishopFox/fix/dnsc2
Add cleanup to unit tests to avoid excessive disk usage
2025-12-30 21:59:21 -07:00
moloch-- 73cc8c38ff Add cleanup to unit tests to avoid excessive disk usage 2025-12-30 17:47:49 -06:00
Joe ec7e2facbc Merge pull request #2075 from BishopFox/fix/dnsc2
Improvements to DNS C2, Port go-assest.sh to Golang
2025-12-30 15:37:37 -07:00
Joe 4969c2dc51 Merge branch 'master' into fix/dnsc2 2025-12-30 15:36:27 -07:00
moloch-- 94cb60b1f7 Updates to tests and implementation of dns c2 2025-12-30 16:35:37 -06:00
moloch-- 33fcfd54ef Update makefile to make goos/goarch override easier 2025-12-30 13:04:52 -06:00
moloch-- 10bb248e50 Code cleanup 2025-12-30 12:18:05 -06:00
moloch-- e023fd507e Port go-assets.sh to golang with go run 2025-12-30 11:55:56 -06:00
moloch-- 1a9f0f38dd Port go-assets.sh to golang with go run 2025-12-30 10:56:02 -06:00
moloch-- 02831a80bf Cleanup outputs of go-assets.sh 2025-12-30 10:26:24 -06:00
moloch-- 9ccff1798e Expand and update dns c2 unit tests 2025-12-30 09:46:49 -06:00
Joe cd5ef8b5c6 Merge pull request #2063 from BishopFox/3e11dependabot/npm_and_yarn/docs/sliver-docs/next-15.5.9
Bump next from 15.5.7 to 15.5.9 in /docs/sliver-docs
2025-12-30 08:12:38 -07:00
moloch-- dda1fdbbc2 Update dns unit tests 2025-12-30 08:52:26 -06:00
Joe 3b49df5662 Merge pull request #2067 from BishopFox/350cdependabot/github_actions/actions/upload-artifact-6
Bump actions/upload-artifact from 5 to 6
2025-12-29 21:44:47 -07:00
Joe 32be71ff8e Merge pull request #2069 from BishopFox/0d1adependabot/github_actions/actions/download-artifact-7
Bump actions/download-artifact from 6 to 7
2025-12-29 21:44:33 -07:00
Joe 3f77da1524 Merge pull request #2068 from BishopFox/f7cedependabot/github_actions/actions/cache-5
Bump actions/cache from 4 to 5
2025-12-29 21:44:19 -07:00
Joe bae3e9e45d Merge pull request #2074 from BishopFox/polite-zig
Update zig download to be more polite and secure issue #2065
2025-12-29 14:52:04 -07:00
moloch-- 205f5b4196 Update zig download to be more polite and secure issue #2065 2025-12-29 12:04:27 -06:00
dependabot[bot] fae7e211a7 Bump actions/download-artifact from 6 to 7
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-15 03:02:52 +00:00
dependabot[bot] 346da14431 Bump actions/cache from 4 to 5
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-15 03:02:47 +00:00
dependabot[bot] 15174f6c31 Bump actions/upload-artifact from 5 to 6
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-15 03:02:44 +00:00
dependabot[bot] f29df040bb Bump next from 15.5.7 to 15.5.9 in /docs/sliver-docs
Bumps [next](https://github.com/vercel/next.js) from 15.5.7 to 15.5.9.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.7...v15.5.9)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.5.9
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-12 01:53:24 +00:00
Joe 0bbecc16ec Merge pull request #2060 from BishopFox/go/v1.25.5
Go v1.25.5
2025-12-05 13:29:26 -07:00
moloch-- 0c216161c2 npm update 2025-12-05 12:28:39 -08:00
moloch-- 8ad9c62eb5 Go v1.25.5 2025-12-04 20:12:24 -08:00
Joe 4c99fbfd82 Merge pull request #2059 from BishopFox/e722dependabot/npm_and_yarn/docs/sliver-docs/next-15.5.7
Bump next from 15.5.4 to 15.5.7 in /docs/sliver-docs
2025-12-04 15:48:34 -07:00
dependabot[bot] ed713e199c Bump next from 15.5.4 to 15.5.7 in /docs/sliver-docs
Bumps [next](https://github.com/vercel/next.js) from 15.5.4 to 15.5.7.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.4...v15.5.7)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.5.7
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-04 20:55:48 +00:00
Joe aacdb71c4a Merge pull request #2058 from BishopFox/e72edependabot/npm_and_yarn/docs/sliver-docs/mdast-util-to-hast-13.2.1
Bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /docs/sliver-docs
2025-12-04 13:54:17 -07:00
dependabot[bot] 6968c3aae5 Bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /docs/sliver-docs
Bumps [mdast-util-to-hast](https://github.com/syntax-tree/mdast-util-to-hast) from 13.2.0 to 13.2.1.
- [Release notes](https://github.com/syntax-tree/mdast-util-to-hast/releases)
- [Commits](https://github.com/syntax-tree/mdast-util-to-hast/compare/13.2.0...13.2.1)

---
updated-dependencies:
- dependency-name: mdast-util-to-hast
  dependency-version: 13.2.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-12-02 04:02:29 +00:00
Joe 410fd93025 Merge pull request #2052 from BishopFox/d161dependabot/github_actions/actions/checkout-6
Bump actions/checkout from 5 to 6
2025-11-23 22:11:44 -07:00
dependabot[bot] f2a0435dd2 Bump actions/checkout from 5 to 6
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-11-24 03:04:10 +00:00
Joe d66ec956a4 Merge pull request #2037 from BishopFox/2979dependabot/go_modules/tailscale.com-1.90.6
Bump tailscale.com from 1.90.2 to 1.90.6
2025-11-03 12:57:04 -07:00
dependabot[bot] 8229722735 Bump tailscale.com from 1.90.2 to 1.90.6
Bumps [tailscale.com](https://github.com/tailscale/tailscale) from 1.90.2 to 1.90.6.
- [Release notes](https://github.com/tailscale/tailscale/releases)
- [Commits](https://github.com/tailscale/tailscale/compare/v1.90.2...v1.90.6)

---
updated-dependencies:
- dependency-name: tailscale.com
  dependency-version: 1.90.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-11-03 03:04:28 +00:00
Joe dcd81a4db7 Merge pull request #2035 from BishopFox/util/rand
Secure Rand Utils
2025-11-02 10:01:16 -07:00
Joe 2617387417 Merge pull request #2028 from BishopFox/586cdependabot/go_modules/tailscale.com-1.90.2
Bump tailscale.com from 1.88.4 to 1.90.2
2025-11-02 09:39:38 -07:00
dependabot[bot] a5861c43f2 Bump tailscale.com from 1.88.4 to 1.90.2
Bumps [tailscale.com](https://github.com/tailscale/tailscale) from 1.88.4 to 1.90.2.
- [Release notes](https://github.com/tailscale/tailscale/releases)
- [Commits](https://github.com/tailscale/tailscale/compare/v1.88.4...v1.90.2)

---
updated-dependencies:
- dependency-name: tailscale.com
  dependency-version: 1.90.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2025-10-27 03:04:45 +00:00
1614 changed files with 67106 additions and 265381 deletions

No files matched your search

+5 -5
View File
@@ -22,7 +22,7 @@ jobs:
git build-essential zlib1g zlib1g-dev wget zip unzip
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Git Fetch Tags
run: git fetch --prune --unshallow --tags -f
@@ -31,7 +31,7 @@ jobs:
run: make servers
- name: Server Artifacts
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: servers
path: ./sliver-server_*
@@ -48,7 +48,7 @@ jobs:
go-version: "^1.25"
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Git Fetch Tags
run: git fetch --prune --unshallow --tags -f
@@ -57,7 +57,7 @@ jobs:
run: make clients
- name: Client Artifacts
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: clients
path: ./sliver-client_*
@@ -72,7 +72,7 @@ jobs:
steps:
- run: mkdir -p ./builds
- uses: actions/download-artifact@v6
- uses: actions/download-artifact@v7
with:
path: ./builds
+2 -2
View File
@@ -22,7 +22,7 @@ jobs:
go-version: "^1.25"
- name: Checkout repository
uses: actions/checkout@v5
uses: actions/checkout@v6
with:
# Must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head of the pull request.
@@ -36,7 +36,7 @@ jobs:
git build-essential zlib1g zlib1g-dev wget zip unzip
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Detect package manager
id: detect-package-manager
@@ -66,7 +66,7 @@ jobs:
uses: actions/configure-pages@v5
- name: Restore cache
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: |
.next/cache
+3 -3
View File
@@ -34,7 +34,7 @@ jobs:
git build-essential zlib1g zlib1g-dev wget zip unzip
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Git Fetch Tags
run: git fetch --prune --unshallow --tags -f
@@ -63,7 +63,7 @@ jobs:
go-version: "^1.25"
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Git Fetch Tags
run: git fetch --prune --unshallow --tags -f
@@ -88,7 +88,7 @@ jobs:
go-version: "^1.25"
- name: Check Out Code
uses: actions/checkout@v5
uses: actions/checkout@v6
- name: Git Fetch Tags
run: git fetch --prune --unshallow --tags -f
+29 -11
View File
@@ -1,23 +1,41 @@
# Repository Guidelines
## Project Structure & Module Organization
Sliver is a Go monorepo. Core client code lives in `client/` (CLI, assets, transports) and server code in `server/` (C2 services, assets, configs). `implant/` holds implant payload sources, while `util/` provides shared helpers that must stay server-side. Protocol buffers live under `protobuf/`, and documentation assets are in `docs/`. Keep generated artifacts (e.g., `sliver-client`, `sliver-server`, `.downloaded_assets`) out of commits.
- `client/` holds the CLI client and interactive commands.
- `server/` contains the server runtime, C2 logic, configs, and web assets.
- `implant/` is the implant runtime and transport implementations.
- `protobuf/` stores protobuf definitions and generated Go types.
- `util/` is shared helpers (crypto, encoders, minisign, etc.).
- `docs/sliver-docs/` is the Next.js documentation site and prebuild scripts.
- `vendor/` is the vendored Go dependency tree used by builds.
- Build outputs land in the repo root as `sliver-client*` and `sliver-server*`.
## Build, Test, and Development Commands
Use `make` targets to stay consistent:
- `make` (or `make default`) compiles client/server binaries with vendored modules and tag defaults.
- `make client` and `make linux-amd64` build platform-specific emitters; prefer `GOOS/GOARCH` env vars over editing sources.
- `./go-assets.sh` refreshes client asset bundles; rerun when static files change.
Regenerate protobuf bindings with `make pb`. For quick checks, `GOFLAGS="-tags client" go build ./client/...` is acceptable, but ensure the matching `server` build also succeeds.
- `make` builds server + client using vendored modules and runs `go-assets.sh`.
- `make client` builds a CGO-free client; `make debug` keeps symbols.
- `make pb` regenerates protobuf stubs from `protobuf/`.
- `./go-tests.sh` runs the curated Go test suite with build tags.
- Docs site: `cd docs/sliver-docs && npm install && npm run dev` for local dev; `npm run build` for a static build.
- Go toolchain: the Makefile enforces Go >= 1.24.
## Coding Style & Naming Conventions
Target Go 1.22+ (see `Makefile` validation). Format code with `gofmt` or `goimports` before committing. Keep package imports segregated (stdlib, third-party, internal). Use lower_snake filenames for assets, PascalCase for exported Go symbols, and prefer explicit names (`sessionID`, `operatorConfig`) over abbreviations. Never import `server` packages inside `client`. Avoid introducing CGO unless justified for cross-platform support.
- Run `gofmt` on all Go code; keep package names lowercase and idiomatic.
- Avoid `CGO` and empty interfaces; prefer explicit types and interfaces.
- Do not import `server` packages from `client`.
- If `math/rand` is required, import it as `insecureRand` and avoid security use.
## Testing Guidelines
Run targeted unit tests with Go’s tooling and project tags, e.g. `go test -tags=client,go_sqlite ./client/command/...`. Full coverage uses `./build.py`; expect long runtimes and ensure ample CPU/RAM. Add tests beside code (`*_test.go`) and mirror package paths. When adding transports or implants, verify integration logs under `~/.sliver/logs/` for regressions and clean up fixtures.
- Go tests follow standard `_test.go` naming within each package.
- Prefer `docker build --target test --build-arg GO_TESTS_FLAGS=--skip-generate .`
- Add or update tests when behavior changes; some server packages run longer tests.
## Commit & Pull Request Guidelines
Sign every commit (`git commit -S`) and write imperative, scoped messages ("Fix DNS canary retry" rather than status updates). Keep vendor updates isolated in their own commit. Pull requests should describe the intent, affected modules, testing performed, and reference GitHub issues. Include before/after screenshots only when UI behavior changes (client console). Ensure CI passes and note any required follow-up.
- Commits must be signed for PRs to `master`; work in a feature branch.
- Commit subjects are short and imperative (e.g., "Fix ...", "Add ...", "Update ...", "Bump ...").
- Changes to `vendor/` should be in a distinct commit.
- PRs should link the tracking issue, note tests run, and confirm `gofmt` plus unit tests as applicable.
## Security & Configuration Tips
Default to secure settings: avoid `math/rand` for secrets, prefer vetted crypto (AES-GCM, Curve25519) per `CONTRIBUTING.md`. Lock down file permissions for generated assets and redact secrets from logs. When touching configuration, favor secure defaults even if it means additional setup for operators.
## Security Notes
- Default to secure configuration; fail closed rather than open.
- Avoid insecure algorithms (MD5/SHA1/AES-ECB); prefer AES-GCM and modern curves.
- Treat user input as untrusted: canonicalize paths and use restrictive file permissions.
+5 -1
View File
@@ -5,6 +5,7 @@
#
# For unit testing:
# docker build --target test .
# docker build --target test --build-arg GO_TESTS_FLAGS=--skip-generate .
#
# STAGE: base
@@ -31,6 +32,9 @@ RUN cp -vv sliver-server /opt/sliver-server
## Use `--target test` in the docker build command to run this stage
FROM base AS test
ARG GO_TESTS_FLAGS=""
ENV GO_TESTS_FLAGS="${GO_TESTS_FLAGS}"
RUN apt-get update --fix-missing \
&& apt-get -y upgrade \
&& apt-get -y install \
@@ -39,7 +43,7 @@ RUN apt-get update --fix-missing \
RUN /opt/sliver-server unpack --force
### Run unit tests
RUN /go/src/github.com/bishopfox/sliver/go-tests.sh
RUN /go/src/github.com/bishopfox/sliver/go-tests.sh ${GO_TESTS_FLAGS}
# STAGE: production
## Final dockerized form of Sliver
+9 -6
View File
@@ -16,7 +16,7 @@ endif
# Prerequisites
#
# https://stackoverflow.com/questions/5618615/check-if-a-program-exists-from-a-makefile
EXECUTABLES = uname sed git zip date cut $(GO)
EXECUTABLES = uname sed git date cut $(GO)
K := $(foreach exec,$(EXECUTABLES),\
$(if $(shell which $(exec)),some string,$(error "No $(exec) in PATH")))
@@ -83,7 +83,9 @@ endif
# If no target is specified, determine GOARCH
ifeq ($(UNAME_P),arm)
ifeq ($(MAKECMDGOALS), )
ENV += GOARCH=arm64
ifeq ($(origin GOARCH), undefined)
ENV += GOARCH=arm64
endif
endif
endif
@@ -105,9 +107,10 @@ endif
# Targets
#
.PHONY: default
default: clean .downloaded_assets validate-go-version
$(ENV) CGO_ENABLED=$(CGO_ENABLED) $(GO) build -mod=vendor -trimpath $(TAGS),server $(LDFLAGS) -o sliver-server$(ARTIFACT_SUFFIX) ./server
$(ENV) CGO_ENABLED=0 $(GO) build -mod=vendor -trimpath $(TAGS),client $(LDFLAGS) -o sliver-client$(ARTIFACT_SUFFIX) ./client
default: clean validate-go-version
env -u GOOS -u GOARCH $(MAKE) GOOS= GOARCH= .downloaded_assets
$(ENV) $(if $(GOOS),GOOS=$(GOOS)) $(if $(GOARCH),GOARCH=$(GOARCH)) CGO_ENABLED=$(CGO_ENABLED) $(GO) build -mod=vendor -trimpath $(TAGS),server $(LDFLAGS) -o sliver-server$(ARTIFACT_SUFFIX) ./server
$(ENV) $(if $(GOOS),GOOS=$(GOOS)) $(if $(GOARCH),GOARCH=$(GOARCH)) CGO_ENABLED=0 $(GO) build -mod=vendor -trimpath $(TAGS),client $(LDFLAGS) -o sliver-client$(ARTIFACT_SUFFIX) ./client
# Allows you to build a CGO-free client for any target e.g. `GOOS=windows GOARCH=arm64 make client`
# NOTE: WireGuard is not supported on all platforms, but most 64-bit GOOS/GOARCH combinations should work.
@@ -199,5 +202,5 @@ clean:
rm -f sliver-client sliver-client_* sliver-server sliver-server_* sliver-*.exe
.downloaded_assets:
./go-assets.sh
$(ENV) $(GO) run -mod=vendor ./util/cmd/assets
touch ./.downloaded_assets
-135
View File
@@ -1,135 +0,0 @@
#!/usr/bin/env python3
"""
Sliver Implant Framework
Copyright (C) 2019 Bishop Fox
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
"""
import os
import sys
import argparse
INFO = "\033[1m\033[36m[*]\033[0m "
WARN = "\033[1m\033[31m[!]\033[0m "
BOLD = "\033[1m"
NORM = "\033[0m"
def exec_cmd(cmd, ignore_status=False):
"""Executes a command with some sugar"""
print("\n" + INFO + " %s\n" % cmd)
status = os.system(cmd)
if status != 0 and not ignore_status:
print("\n" + WARN + "Command did not exit cleanly (%s) " % status)
if input("continue? [Y/n]: ").upper() != "Y":
sys.exit(status)
return status
def docker_rm_volumes():
exec_cmd("docker volume rm $(docker volume ls -q)", ignore_status=True)
def docker_rm_containers():
exec_cmd("docker rm $(docker ps -a -q)", ignore_status=True)
def docker_rm_images():
exec_cmd("docker rmi -f $(docker images -q)", ignore_status=True)
def docker_prune():
exec_cmd("docker image prune -f", ignore_status=True)
def docker_rm_all():
docker_rm_containers()
docker_rm_images()
docker_rm_volumes()
def build():
status = exec_cmd("docker build --target test .")
if status == 0:
print("\n\n" + INFO + "Build successful")
def main(args):
"""Execute the respective task(s) based on cli args"""
if args.rm_images:
docker_rm_images()
if args.rm_containers:
docker_rm_containers()
if args.rm_volumes:
docker_rm_volumes()
if args.rm_all:
docker_rm_all()
if not args.no_prune:
docker_prune()
if not args.no_build:
build()
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="automates interactions with docker")
parser.add_argument(
"--no-build",
"-b",
help="do not run build",
dest="no_build",
action="store_true",
)
parser.add_argument(
"--no-prune",
"-np",
help="do not prune images",
dest="no_prune",
action="store_true",
)
parser.add_argument(
"--rm-all",
"-rma",
help="rm all containers, images, and volumes",
dest="rm_all",
action="store_true",
)
parser.add_argument(
"--rm-containers",
"-rmc",
help="rm all containers",
dest="rm_containers",
action="store_true",
)
parser.add_argument(
"--rm-images",
"-rmi",
help="rm all images",
dest="rm_images",
action="store_true",
)
parser.add_argument(
"--rm-volumes",
"-rmv",
help="rm dangling volumes only",
dest="rm_volumes",
action="store_true",
)
try:
main(parser.parse_args())
except KeyboardInterrupt:
print("\n\n" + WARN + "User stopped process")
+69 -1
View File
@@ -35,6 +35,7 @@ import (
"github.com/AlecAivazis/survey/v2"
"github.com/bishopfox/sliver/client/console"
consts "github.com/bishopfox/sliver/client/constants"
"github.com/bishopfox/sliver/client/forms"
"github.com/bishopfox/sliver/client/spin"
"github.com/bishopfox/sliver/protobuf/clientpb"
"github.com/bishopfox/sliver/protobuf/commonpb"
@@ -89,6 +90,21 @@ var (
// GenerateCmd - The main command used to generate implant binaries
func GenerateCmd(cmd *cobra.Command, con *console.SliverClient, args []string) {
if shouldRunGenerateForm(cmd, con, args) {
result, err := forms.GenerateForm()
if err != nil {
if errors.Is(err, forms.ErrUserAborted) {
return
}
con.PrintErrorf("Generate form failed: %s\n", err)
return
}
if err := applyGenerateForm(cmd, result); err != nil {
con.PrintErrorf("Failed to apply generate form values: %s\n", err)
return
}
}
name, config := parseCompileFlags(cmd, con)
if config == nil {
return
@@ -117,6 +133,58 @@ func GenerateCmd(cmd *cobra.Command, con *console.SliverClient, args []string) {
}
}
func shouldRunGenerateForm(cmd *cobra.Command, con *console.SliverClient, args []string) bool {
if con == nil || con.IsCLI {
return false
}
if len(args) != 0 {
return false
}
return cmd.Flags().NFlag() == 0
}
func applyGenerateForm(cmd *cobra.Command, result *forms.GenerateFormResult) error {
if err := cmd.Flags().Set("os", result.OS); err != nil {
return err
}
if err := cmd.Flags().Set("arch", result.Arch); err != nil {
return err
}
if err := cmd.Flags().Set("format", result.Format); err != nil {
return err
}
name := strings.TrimSpace(result.Name)
if name != "" {
if err := cmd.Flags().Set("name", name); err != nil {
return err
}
}
save := strings.TrimSpace(result.Save)
if save != "" {
if err := cmd.Flags().Set("save", save); err != nil {
return err
}
}
c2Value := strings.TrimSpace(result.C2Value)
switch result.C2Type {
case "mtls":
return cmd.Flags().Set("mtls", c2Value)
case "wg":
return cmd.Flags().Set("wg", c2Value)
case "http":
return cmd.Flags().Set("http", c2Value)
case "dns":
return cmd.Flags().Set("dns", c2Value)
case "named-pipe":
return cmd.Flags().Set("named-pipe", c2Value)
case "tcp-pivot":
return cmd.Flags().Set("tcp-pivot", c2Value)
default:
return errors.New("unsupported C2 transport selection")
}
}
func expandPath(path string) string {
// unless path starts with ~
if len(path) == 0 || path[0] != 126 {
@@ -576,7 +644,7 @@ func hasValidC2AdvancedOptions(options url.Values) (bool, error) {
if testValue != "wininet" {
return false, fmt.Errorf("C2 option \"driver\" must be empty for the default driver or \"wininet\" for the wininet driver (Windows only)")
}
case "force-http", "disable-accept-header", "disable-upgrade-header", "ask-proxy-creds", "force-base32":
case "force-http", "disable-accept-header", "disable-upgrade-header", "ask-proxy-creds":
if testValue != "true" && testValue != "false" {
return false, fmt.Errorf("C2 option \"%s\" must be a boolean value: true or false", key)
}
+133
View File
@@ -0,0 +1,133 @@
package forms
import (
"errors"
"strings"
"github.com/bishopfox/sliver/util"
"github.com/charmbracelet/huh"
)
// ErrUserAborted exposes the huh user-abort sentinel for callers.
var ErrUserAborted = huh.ErrUserAborted
// GenerateFormResult captures the inputs needed to drive the generate command.
type GenerateFormResult struct {
OS string
Arch string
Format string
Name string
C2Type string
C2Value string
Save string
}
// GenerateForm prompts for core generate flags and returns the collected values.
func GenerateForm() (*GenerateFormResult, error) {
result := &GenerateFormResult{
OS: "windows",
Arch: "amd64",
Format: "exe",
C2Type: "mtls",
}
form := huh.NewForm(
huh.NewGroup(
huh.NewSelect[string]().
Title("Target operating system").
Options(
huh.NewOption("Windows", "windows"),
huh.NewOption("Linux", "linux"),
huh.NewOption("macOS", "darwin"),
).
Value(&result.OS),
huh.NewSelect[string]().
Title("CPU architecture").
OptionsFunc(func() []huh.Option[string] {
switch result.OS {
case "darwin":
return []huh.Option[string]{
huh.NewOption("amd64", "amd64"),
huh.NewOption("arm64", "arm64"),
}
case "linux":
return []huh.Option[string]{
huh.NewOption("amd64", "amd64"),
huh.NewOption("386", "386"),
}
default:
return []huh.Option[string]{
huh.NewOption("amd64", "amd64"),
huh.NewOption("386", "386"),
}
}
}, &result.OS).
Value(&result.Arch),
huh.NewSelect[string]().
Title("Output format").
OptionsFunc(func() []huh.Option[string] {
options := []huh.Option[string]{
huh.NewOption("Executable", "exe"),
huh.NewOption("Shared library", "shared"),
}
if result.OS == "windows" {
options = append(options,
huh.NewOption("Service", "service"),
huh.NewOption("Shellcode", "shellcode"),
)
}
return options
}, &result.OS).
Value(&result.Format),
),
huh.NewGroup(
huh.NewInput().
Title("Implant name (optional)").
Value(&result.Name).
Validate(func(value string) error {
if strings.TrimSpace(value) == "" {
return nil
}
return util.AllowedName(strings.ToLower(value))
}),
huh.NewSelect[string]().
Title("C2 transport").
OptionsFunc(func() []huh.Option[string] {
options := []huh.Option[string]{
huh.NewOption("mTLS", "mtls"),
huh.NewOption("WireGuard", "wg"),
huh.NewOption("HTTP(S)", "http"),
huh.NewOption("DNS", "dns"),
}
if result.OS == "windows" {
options = append(options,
huh.NewOption("Named pipe", "named-pipe"),
huh.NewOption("TCP pivot", "tcp-pivot"),
)
}
return options
}, &result.OS).
Value(&result.C2Type),
huh.NewInput().
Title("C2 connection string(s)").
Description("Comma separated; example: 1.2.3.4:8888 or https://c2.example.com").
Value(&result.C2Value).
Validate(func(value string) error {
if strings.TrimSpace(value) == "" {
return errors.New("connection string required")
}
return nil
}),
huh.NewInput().
Title("Save path (optional)").
Description("File or directory; defaults to the current directory").
Value(&result.Save),
),
)
if err := form.Run(); err != nil {
return nil, err
}
return result, nil
}
+692 -637
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -26,7 +26,7 @@
"framer-motion": "^12.23.22",
"fuse.js": "^7.1.0",
"lunr": "^2.3.9",
"next": "^15.5.4",
"next": "^15.5.9",
"next-themes": "^0.4.6",
"react": "^19.2.0",
"react-dom": "^19.2.0",
@@ -29,6 +29,5 @@ generate --http http://example.com?driver=wininet
- `retry-count` - Upon query failure, retry this many times before failing.
- `workers-per-resolver` - Number of worker goroutines per functional DNS resolver.
- `max-errors` - Max number of query errors before failing (integer parsed by `strconv.Atoi`).
- `force-base32` - Always use Base 32 encoding.
- `force-resolv-conf` - Force the use of a provided `resolv.conf`. Note that you'll need to URL encode the newlines/etc of the file contents into the parameter.
- `resolvers` - Force the use of specific DNS resolvers. You can supply more than one by separating them with `+`, i.e. `...?resolvers=1.1.1.1+9.9.9.9`
-312
View File
@@ -1,312 +0,0 @@
#!/bin/bash
# Sliver Implant Framework
# Copyright (C) 2019 Bishop Fox
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
set -e
# Creates the static go asset archives
GO_VER="1.25.3"
GARBLE_VER="1.25.3"
ZIG_VER="0.15.1"
# Zig significantly throttles downloads from the main site, so we use
# community mirrors. We fetch the list of mirrors at runtime, but
# fall back to a set of default mirrors if the fetch fails.
# We also use the official public key for verifying Zig downloads from mirrors.
ZIG_MINISIGN_PUBKEY="RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U"
DEFAULT_ZIG_MIRRORS=(
"https://pkg.machengine.org/zig"
"https://zigmirror.hryx.net/zig"
"https://zig.linus.dev/zig"
"https://zig.squirl.dev"
"https://zig.florent.dev"
"https://zig.mirror.mschae23.de/zig"
"https://zigmirror.meox.dev"
)
ZIG_MIRRORS=()
MINISIGN_HELPER=""
MINISIGN_HELPER_DIR=""
BLOAT_FILES="AUTHORS CONTRIBUTORS PATENTS VERSION favicon.ico robots.txt SECURITY.md CONTRIBUTING.md README.md ./doc ./test ./api ./misc"
if ! [ -x "$(command -v curl)" ]; then
echo 'Error: curl is not installed.' >&2
exit 1
fi
if ! [ -x "$(command -v zip)" ]; then
echo 'Error: zip is not installed.' >&2
exit 1
fi
if ! [ -x "$(command -v unzip)" ]; then
echo 'Error: unzip is not installed.' >&2
exit 1
fi
if ! [ -x "$(command -v tar)" ]; then
echo 'Error: tar is not installed.' >&2
exit 1
fi
if ! [ -x "$(command -v go)" ]; then
echo 'Error: go is not installed.' >&2
exit 1
fi
REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
OUTPUT_DIR=$REPO_DIR/server/assets/fs
mkdir -p $OUTPUT_DIR
WORK_DIR=`mktemp -d`
echo "-----------------------------------------------------------------"
echo "$WORK_DIR (Output: $OUTPUT_DIR)"
echo "-----------------------------------------------------------------"
cd $WORK_DIR
# Load official Zig mirror list with a fallback to bundled defaults.
load_zig_mirrors() {
local mirrors_text
local -a mirrors=()
if mirrors_text=$(curl -sS -L --fail https://ziglang.org/download/community-mirrors.txt 2>/dev/null); then
while IFS= read -r line; do
line="${line%%#*}"
line="$(printf '%s' "$line" | sed -e 's/[[:space:]]*$//' -e 's/^[[:space:]]*//')"
if [[ -n "$line" ]]; then
mirrors+=("$line")
fi
done <<< "$mirrors_text"
fi
if [ "${#mirrors[@]}" -eq 0 ]; then
mirrors=("${DEFAULT_ZIG_MIRRORS[@]}")
fi
ZIG_MIRRORS=("${mirrors[@]}")
}
# Shuffle mirror list for each download attempt to spread load.
randomized_mirrors() {
if [ "${#ZIG_MIRRORS[@]}" -eq 0 ]; then
load_zig_mirrors
fi
local -a shuffled=("${ZIG_MIRRORS[@]}")
local i j temp
for ((i=${#shuffled[@]}-1; i>0; i--)); do
j=$((RANDOM % (i + 1)))
temp=${shuffled[i]}
shuffled[i]=${shuffled[j]}
shuffled[j]=$temp
done
printf '%s\n' "${shuffled[@]}"
}
# Build a temporary helper that verifies signatures via util/minisign.
ensure_minisign_helper() {
if [ -n "$MINISIGN_HELPER" ] && [ -x "$MINISIGN_HELPER" ]; then
return
fi
local helper_dir
helper_dir=$(mktemp -d "$REPO_DIR/.zig-verify.XXXXXX")
if ! (cd "$REPO_DIR" && GO111MODULE=on GOFLAGS=-mod=vendor go build -o "$helper_dir/zig-verify" ./util/cmd/zigverify); then
echo 'Error: unable to build minisign verification helper.' >&2
exit 1
fi
MINISIGN_HELPER="$helper_dir/zig-verify"
MINISIGN_HELPER_DIR="$helper_dir"
trap 'rm -rf "$MINISIGN_HELPER_DIR"' EXIT
}
verify_minisig() {
ensure_minisign_helper
ZIG_PUBLIC_KEY="$ZIG_MINISIGN_PUBKEY" "$MINISIGN_HELPER" "$1" "$2"
}
download_zig() {
local platform="$1"
local arch="$2"
local remote_name="$3"
local local_name="$4"
local dest_dir="$OUTPUT_DIR/$platform/$arch"
local dest_path="$dest_dir/$local_name"
local minisig_url="https://ziglang.org/download/$ZIG_VER/$remote_name.minisig"
local mirror_url
local success=0
mkdir -p "$dest_dir"
rm -f "$dest_path"
while read -r mirror; do
mirror_url="${mirror%/}/$ZIG_VER/$remote_name"
echo "Attempting Zig download from $mirror_url"
local tmp_tar
local tmp_sig
tmp_tar=$(mktemp)
tmp_sig=$(mktemp)
local verification_failed=0
if curl -L --fail --output "$tmp_tar" "$mirror_url"; then
if curl -L --fail --output "$tmp_sig" "$minisig_url"; then
if verify_minisig "$tmp_tar" "$tmp_sig"; then
mv "$tmp_tar" "$dest_path"
rm -f "$tmp_sig"
echo "Downloaded and verified Zig package -> $dest_path"
success=1
break
else
echo "[!] Signature verification failed for $mirror_url" >&2
verification_failed=1
fi
fi
fi
if [ "$verification_failed" -eq 1 ]; then
echo "[!] Deleting corrupted download $tmp_tar" >&2
fi
rm -f "$tmp_tar" "$tmp_sig"
echo "[!] Failed to download or verify Zig from $mirror_url" >&2
done < <(randomized_mirrors)
if [ "$success" -ne 1 ]; then
echo "[!] Error: unable to download and verify Zig package $remote_name" >&2
exit 1
fi
}
# --- Darwin (amd64) ---
curl --output go$GO_VER.darwin-amd64.tar.gz https://dl.google.com/go/go$GO_VER.darwin-amd64.tar.gz
tar xvf go$GO_VER.darwin-amd64.tar.gz
cd go
rm -rf $BLOAT_FILES
zip -r ../src.zip ./src # Zip up /src we only need to do this once
rm -rf ./src
rm -f ./pkg/tool/darwin_amd64/doc
rm -f ./pkg/tool/darwin_amd64/tour
rm -f ./pkg/tool/darwin_amd64/test2json
cd ..
cp -vv src.zip $OUTPUT_DIR/src.zip
rm -f src.zip
zip -r darwin-go.zip ./go
mkdir -p $OUTPUT_DIR/darwin/amd64
cp -vv darwin-go.zip $OUTPUT_DIR/darwin/amd64/go.zip
rm -rf ./go
rm -f darwin-go.zip go$GO_VER.darwin-amd64.tar.gz
# --- Darwin (arm64) ---
curl --output go$GO_VER.darwin-arm64.tar.gz https://dl.google.com/go/go$GO_VER.darwin-arm64.tar.gz
tar xvf go$GO_VER.darwin-arm64.tar.gz
cd go
rm -rf $BLOAT_FILES
zip -r ../src.zip ./src # Zip up /src we only need to do this once
rm -rf ./src
rm -f ./pkg/tool/darwin_arm64/doc
rm -f ./pkg/tool/darwin_arm64/tour
rm -f ./pkg/tool/darwin_arm64/test2json
cd ..
cp -vv src.zip $OUTPUT_DIR/src.zip
rm -f src.zip
zip -r darwin-go.zip ./go
mkdir -p $OUTPUT_DIR/darwin/arm64
cp -vv darwin-go.zip $OUTPUT_DIR/darwin/arm64/go.zip
rm -rf ./go
rm -f darwin-go.zip go$GO_VER.darwin-arm64.tar.gz
# --- Linux (amd64) ---
curl --output go$GO_VER.linux-amd64.tar.gz https://dl.google.com/go/go$GO_VER.linux-amd64.tar.gz
tar xvf go$GO_VER.linux-amd64.tar.gz
cd go
rm -rf $BLOAT_FILES
rm -rf ./src
rm -f ./pkg/tool/linux_amd64/doc
rm -f ./pkg/tool/linux_amd64/tour
rm -f ./pkg/tool/linux_amd64/test2json
cd ..
zip -r linux-go.zip ./go
mkdir -p $OUTPUT_DIR/linux/amd64
cp -vv linux-go.zip $OUTPUT_DIR/linux/amd64/go.zip
rm -rf ./go
rm -f linux-go.zip go$GO_VER.linux-amd64.tar.gz
# --- Linux (arm64) ---
curl --output go$GO_VER.linux-arm64.tar.gz https://dl.google.com/go/go$GO_VER.linux-arm64.tar.gz
tar xvf go$GO_VER.linux-arm64.tar.gz
cd go
rm -rf $BLOAT_FILES
rm -rf ./src
rm -f ./pkg/tool/linux_arm64/doc
rm -f ./pkg/tool/linux_arm64/tour
rm -f ./pkg/tool/linux_arm64/test2json
cd ..
zip -r linux-go.zip ./go
mkdir -p $OUTPUT_DIR/linux/arm64
cp -vv linux-go.zip $OUTPUT_DIR/linux/arm64/go.zip
rm -rf ./go
rm -f linux-go.zip go$GO_VER.linux-arm64.tar.gz
# --- Windows ---
curl --output go$GO_VER.windows-amd64.zip https://dl.google.com/go/go$GO_VER.windows-amd64.zip
unzip go$GO_VER.windows-amd64.zip
cd go
rm -rf $BLOAT_FILES
rm -rf ./src
rm -f ./pkg/tool/windows_amd64/doc.exe
rm -f ./pkg/tool/windows_amd64/tour.exe
rm -f ./pkg/tool/windows_amd64/test2json.exe
cd ..
zip -r windows-go.zip ./go
mkdir -p $OUTPUT_DIR/windows/amd64
cp -vv windows-go.zip $OUTPUT_DIR/windows/amd64/go.zip
rm -rf ./go
rm -f windows-go.zip go$GO_VER.windows-amd64.zip
echo "-----------------------------------------------------------------"
echo " Zig"
echo "-----------------------------------------------------------------"
download_zig "darwin" "amd64" "zig-x86_64-macos-$ZIG_VER.tar.xz" "zig.tar.xz"
download_zig "darwin" "arm64" "zig-aarch64-macos-$ZIG_VER.tar.xz" "zig.tar.xz"
download_zig "linux" "amd64" "zig-x86_64-linux-$ZIG_VER.tar.xz" "zig.tar.xz"
download_zig "linux" "arm64" "zig-aarch64-linux-$ZIG_VER.tar.xz" "zig.tar.xz"
# Windows ships a zip instead of a tarball
download_zig "windows" "amd64" "zig-x86_64-windows-$ZIG_VER.zip" "zig.zip"
echo "-----------------------------------------------------------------"
echo " Garble"
echo "-----------------------------------------------------------------"
echo "curl -L --fail --output $OUTPUT_DIR/linux/amd64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_linux-amd64"
curl -L --fail --output $OUTPUT_DIR/linux/amd64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_linux-amd64
echo "curl -L --fail --output $OUTPUT_DIR/linux/arm64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_linux-arm64"
curl -L --fail --output $OUTPUT_DIR/linux/arm64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_linux-arm64
echo "curl -L --fail --output $OUTPUT_DIR/windows/amd64/garble.exe https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_windows-amd64.exe"
curl -L --fail --output $OUTPUT_DIR/windows/amd64/garble.exe https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_windows-amd64.exe
echo "curl -L --fail --output $OUTPUT_DIR/darwin/amd64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_darwin-amd64"
curl -L --fail --output $OUTPUT_DIR/darwin/amd64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_darwin-amd64
echo "curl -L --fail --output $OUTPUT_DIR/darwin/arm64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_darwin-arm64"
curl -L --fail --output $OUTPUT_DIR/darwin/arm64/garble https://github.com/moloch--/garble/releases/download/v$GARBLE_VER/garble_darwin-arm64
# --- Cleanup ---
echo -e "clean up: $WORK_DIR"
rm -rf $WORK_DIR
echo -e "\n[*] All done\n"
+16 -5
View File
@@ -17,6 +17,13 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
SKIP_GENERATE=0
for arg in "$@"; do
if [ "$arg" = "--skip-generate" ]; then
SKIP_GENERATE=1
fi
done
echo "----------------------------------------------------------------"
echo "WARNING: Running unit tests on slow systems can take a LONG time"
echo " Recommended to only run on 16+ CPU cores and 32Gb+ RAM"
@@ -180,10 +187,14 @@ else
fi
# server / generate
export GOPROXY=off
if go test -tags=server,$TAGS ./server/generate -timeout 6h ; then
:
if [ "$SKIP_GENERATE" -eq 0 ]; then
export GOPROXY=off
if go test -tags=server,$TAGS ./server/generate -timeout 6h ; then
:
else
cat ~/.sliver/logs/sliver.log
exit 1
fi
else
cat ~/.sliver/logs/sliver.log
exit 1
echo "Skipping ./server/generate tests (--skip-generate)"
fi
+24 -20
View File
@@ -1,6 +1,6 @@
module github.com/bishopfox/sliver
go 1.25.1
go 1.25.3
replace github.com/rsteube/carapace v0.36.3 => github.com/reeflective/carapace v0.46.3-0.20231214181515-27e49f3c3b69
@@ -13,6 +13,9 @@ require (
github.com/Binject/universal v0.0.0-20220519011857-bea739e758c0
github.com/Ne0nd0g/go-clr v1.0.3
github.com/alecthomas/chroma v0.10.0
github.com/charmbracelet/huh v0.8.0
github.com/charmbracelet/huh/spinner v0.0.0-20251215014908-6f7d32faaff3
github.com/charmbracelet/lipgloss v1.1.0
github.com/cheggaaa/pb/v3 v3.1.7
github.com/chromedp/cdproto v0.0.0-20250803210736-d308e07a266d
github.com/chromedp/chromedp v0.14.1
@@ -63,7 +66,7 @@ require (
gorm.io/gorm v1.31.0
gvisor.dev/gvisor v0.0.0-20250503011706-39ed1f5ac29c
modernc.org/sqlite v1.39.0
tailscale.com v1.88.4
tailscale.com v1.90.6
)
require (
@@ -75,24 +78,23 @@ require (
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/akutz/memconn v0.1.0 // indirect
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/awgh/cppgo v0.0.0-20210224085512-3d24bca8edc0 // indirect
github.com/awgh/rawreader v0.0.0-20200626064944-56820a9c6da4 // indirect
github.com/aws/aws-sdk-go-v2 v1.39.2 // indirect
github.com/aws/aws-sdk-go-v2/config v1.31.12 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.18.16 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.9 // indirect
github.com/aws/aws-sdk-go-v2/service/ssm v1.65.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.29.6 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.38.6 // indirect
github.com/aws/smithy-go v1.23.0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/carapace-sh/carapace v1.9.0 // indirect
github.com/carapace-sh/carapace-shlex v1.1.0 // indirect
github.com/catppuccin/go v0.3.0 // indirect
github.com/charmbracelet/bubbles v0.21.1-0.20250623103423-23b8fd6302d7 // indirect
github.com/charmbracelet/bubbletea v1.3.10 // indirect
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect
github.com/charmbracelet/x/ansi v0.10.1 // indirect
github.com/charmbracelet/x/cellbuf v0.0.13 // indirect
github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/chromedp/sysutil v1.1.0 // indirect
github.com/clipperhouse/uax29/v2 v2.2.0 // indirect
github.com/coder/websocket v1.8.12 // indirect
@@ -100,15 +102,14 @@ require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dblohm7/wingoes v0.0.0-20250822163801-6d8e6105c62d // indirect
github.com/demisto/goxforce v0.0.0-20160322194047-db8357535b1d // indirect
github.com/digitalocean/go-smbios v0.0.0-20180907143718-390a4f403a8e // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/gaissmai/bart v0.25.0 // indirect
github.com/gen2brain/shm v0.1.1 // indirect
github.com/glebarez/go-sqlite v1.22.0 // indirect
github.com/go-json-experiment/json v0.0.0-20250910080747-cc2cfa0554c3 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-sql-driver/mysql v1.9.3 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
@@ -120,7 +121,6 @@ require (
github.com/google/nftables v0.3.0 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hdevalence/ed25519consensus v0.2.0 // indirect
github.com/illarion/gonotify/v3 v3.0.2 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@@ -132,17 +132,21 @@ require (
github.com/jinzhu/now v1.1.5 // indirect
github.com/jsimonetti/rtnetlink v1.4.2 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/lxn/win v0.0.0-20210218163916-a377121e959e // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.19 // indirect
github.com/mdlayher/genetlink v1.3.2 // indirect
github.com/mdlayher/netlink v1.8.0 // indirect
github.com/mdlayher/sdnotify v1.0.0 // indirect
github.com/mdlayher/socket v0.5.1 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect
github.com/moloch--/go-keystone v0.0.2 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/ncruces/julianday v1.0.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
@@ -156,16 +160,16 @@ require (
github.com/tailscale/go-winio v0.0.0-20231025203758-c4f33415bf55 // indirect
github.com/tailscale/goupnp v1.0.1-0.20210804011211-c64d0f06ea05 // indirect
github.com/tailscale/hujson v0.0.0-20250605163823-992244df8c5a // indirect
github.com/tailscale/netlink v1.1.1-0.20240822203006-4d49adab4de7 // indirect
github.com/tailscale/peercred v0.0.0-20250107143737-35a0c7bd7edc // indirect
github.com/tailscale/web-client-prebuilt v0.0.0-20250124233751-d4cd19a26976 // indirect
github.com/tailscale/wireguard-go v0.0.0-20250716170648-1d0488a3d7da // indirect
github.com/thedevsaddam/gojsonq/v2 v2.5.2 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go4.org/mem v0.0.0-20240501181205-ae6ca9944745 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/mod v0.28.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/time v0.13.0 // indirect
golang.org/x/tools v0.37.0 // indirect
+59 -6
View File
@@ -29,6 +29,8 @@ github.com/Binject/universal v0.0.0-20220519011857-bea739e758c0/go.mod h1:J3XDRl
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ=
github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/Ne0nd0g/go-clr v1.0.3 h1:xt92wwuqY23ZSC7RuHD3mKu3K22Bk5NNbxI803vojK4=
@@ -50,6 +52,8 @@ github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e h1:4dAU9FXIyQktp
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z4=
github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI=
github.com/awgh/cppgo v0.0.0-20210224085512-3d24bca8edc0 h1:JjwxKkxzcBk4k8147g0eBQRCIy0UN1Be8AAv6RaIj4Q=
github.com/awgh/cppgo v0.0.0-20210224085512-3d24bca8edc0/go.mod h1:IbERvuyb387Hppp8hX0SQTFt/mkej8+OhuS8L0nC2CI=
github.com/awgh/rawreader v0.0.0-20200626064944-56820a9c6da4 h1:cIAK2NNf2yafdgpFRNJrgZMwvy61BEVpGoHc2n4/yWs=
@@ -82,12 +86,48 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.38.6 h1:p3jIvqYwUZgu/XYeI48bJxOhvm47
github.com/aws/aws-sdk-go-v2/service/sts v1.38.6/go.mod h1:WtKK+ppze5yKPkZ0XwqIVWD4beCwv056ZbPQNoeHqM8=
github.com/aws/smithy-go v1.23.0 h1:8n6I3gXzWJB2DxBDnfxgBaSX6oe0d/t10qGz7OKqMCE=
github.com/aws/smithy-go v1.23.0/go.mod h1:t1ufH5HMublsJYulve2RKmHDC15xu1f26kHCp/HgceI=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
github.com/aymanbagabas/go-udiff v0.3.1 h1:LV+qyBQ2pqe0u42ZsUEtPiCaUoqgA9gYRDs3vj1nolY=
github.com/aymanbagabas/go-udiff v0.3.1/go.mod h1:G0fsKmG+P6ylD0r6N/KgQD/nWzgfnl8ZBcNLgcbrw8E=
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
github.com/carapace-sh/carapace v1.9.0 h1:IuEP6YeeK4fhrH7MoOyL0I3GMhvROerkw6YGxkx8Fbw=
github.com/carapace-sh/carapace v1.9.0/go.mod h1:Zs3DpsawpFm+8fEfRVYE9EzMBYH8u7S3Aw9yAAKX0JM=
github.com/carapace-sh/carapace-shlex v1.1.0 h1:58QSJTUxghpa/rEn6+Z9UM/Hy0ipVHDUn9S1mp4i+ZE=
github.com/carapace-sh/carapace-shlex v1.1.0/go.mod h1:lJ4ZsdxytE0wHJ8Ta9S7Qq0XpjgjU0mdfCqiI2FHx7M=
github.com/catppuccin/go v0.3.0 h1:d+0/YicIq+hSTo5oPuRi5kOpqkVA5tAsU6dNhvRu+aY=
github.com/catppuccin/go v0.3.0/go.mod h1:8IHJuMGaUUjQM82qBrGNBv7LFq6JI3NnQCF6MOlZjpc=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/charmbracelet/bubbles v0.21.1-0.20250623103423-23b8fd6302d7 h1:JFgG/xnwFfbezlUnFMJy0nusZvytYysV4SCS2cYbvws=
github.com/charmbracelet/bubbles v0.21.1-0.20250623103423-23b8fd6302d7/go.mod h1:ISC1gtLcVilLOf23wvTfoQuYbW2q0JevFxPfUzZ9Ybw=
github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw=
github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc h1:4pZI35227imm7yK2bGPcfpFEmuY1gc2YSTShr4iJBfs=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc/go.mod h1:X4/0JoqgTIPSFcRA/P6INZzIuyqdFY5rm8tb41s9okk=
github.com/charmbracelet/huh v0.8.0 h1:Xz/Pm2h64cXQZn/Jvele4J3r7DDiqFCNIVteYukxDvY=
github.com/charmbracelet/huh v0.8.0/go.mod h1:5YVc+SlZ1IhQALxRPpkGwwEKftN/+OlJlnJYlDRFqN4=
github.com/charmbracelet/huh/spinner v0.0.0-20251215014908-6f7d32faaff3 h1:KUeWGoKnmyrLaDIa0smE6pK5eFMZWNIxPGweQR12iLg=
github.com/charmbracelet/huh/spinner v0.0.0-20251215014908-6f7d32faaff3/go.mod h1:OMqKat/mm9a/qOnpuNOPyYO9bPzRNnmzLnRZT5KYltg=
github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY=
github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30=
github.com/charmbracelet/x/ansi v0.10.1 h1:rL3Koar5XvX0pHGfovN03f5cxLbCF2YvLeyz7D2jVDQ=
github.com/charmbracelet/x/ansi v0.10.1/go.mod h1:3RQDQ6lDnROptfpWuUVIUG64bD2g2BgntdxH0Ya5TeE=
github.com/charmbracelet/x/cellbuf v0.0.13 h1:/KBBKHuVRbq1lYx5BzEHBAFBP8VcQzJejZ/IA3iR28k=
github.com/charmbracelet/x/cellbuf v0.0.13/go.mod h1:xe0nKWGd3eJgtqZRaN9RjMtK7xUYchjzPr7q6kcvCCs=
github.com/charmbracelet/x/conpty v0.1.0 h1:4zc8KaIcbiL4mghEON8D72agYtSeIgq8FSThSPQIb+U=
github.com/charmbracelet/x/conpty v0.1.0/go.mod h1:rMFsDJoDwVmiYM10aD4bH2XiRgwI7NYJtQgl5yskjEQ=
github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86 h1:JSt3B+U9iqk37QUU2Rvb6DSBYRLtWqFqfxf8l5hOZUA=
github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86/go.mod h1:2P0UgXMEa6TsToMSuFqKFQR+fZTO9CNGUNokkPatT/0=
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91 h1:payRxjMjKgx2PaCWLZ4p3ro9y97+TVLZNaRZgJwSVDQ=
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 h1:qko3AQ4gK1MTS/de7F5hPGx6/k1u0w4TeYmBFwzYVP4=
github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0/go.mod h1:pBhA0ybfXv6hDjQUZ7hk1lVxBiUbupdw5R31yPUViVQ=
github.com/charmbracelet/x/term v0.2.1 h1:AQeHeLZ1OqSXhrAWpYUtZyX1T3zVxfpZuEQMIQaGIAQ=
github.com/charmbracelet/x/term v0.2.1/go.mod h1:oQ4enTYFV7QN4m0i9mzHrViD7TQKvNEEkHUMCmsxdUg=
github.com/charmbracelet/x/termios v0.1.1 h1:o3Q2bT8eqzGnGPOYheoYS8eEleT5ZVNYNy8JawjaNZY=
github.com/charmbracelet/x/termios v0.1.1/go.mod h1:rB7fnv1TgOPOyyKRJ9o+AsTU/vK5WHJ2ivHeut/Pcwo=
github.com/charmbracelet/x/xpty v0.1.2 h1:Pqmu4TEJ8KeA9uSkISKMU3f+C1F6OGBn8ABuGlqCbtI=
github.com/charmbracelet/x/xpty v0.1.2/go.mod h1:XK2Z0id5rtLWcpeNiMYBccNNBrP2IJnzHI0Lq13Xzq4=
github.com/cheggaaa/pb/v3 v3.1.7 h1:2FsIW307kt7A/rz/ZI2lvPO+v3wKazzE4K/0LtTWsOI=
github.com/cheggaaa/pb/v3 v3.1.7/go.mod h1:/Ji89zfVPeC/u5j8ukD0MBPHt2bzTYp74lQ7KlgFWTQ=
github.com/chromedp/cdproto v0.0.0-20250803210736-d308e07a266d h1:ZtA1sedVbEW7EW80Iz2GR3Ye6PwbJAJXjv7D74xG6HU=
@@ -134,6 +174,8 @@ github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymF
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
github.com/fatih/color v1.12.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
@@ -271,6 +313,8 @@ github.com/lesnuages/go-winio v0.4.19 h1:lFDu1mnhg5em+8zTHO4ChSD11J56xkyE8m3N8Ir
github.com/lesnuages/go-winio v0.4.19/go.mod h1:rm7jf4kBcldxMeljR7c7XY1qVCBc+8z3PtSgkrT9Clk=
github.com/lesnuages/snitch v0.6.0 h1:vOao32MdYDHYDcTAq2CKszDh6Dn8Bv1Gmgc8uVh193U=
github.com/lesnuages/snitch v0.6.0/go.mod h1:RfXlAVJYL7gUoZ9VMMon4LqrPiAjC8x62Ntbse4a4s0=
github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lxn/win v0.0.0-20210218163916-a377121e959e h1:H+t6A/QJMbhCSEH5rAuRxh+CtW96g0Or0Fxa9IKr4uc=
github.com/lxn/win v0.0.0-20210218163916-a377121e959e/go.mod h1:KxxjdtRkfNoYDCUP5ryK7XJJNTnpC8atvtmTheChOtk=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
@@ -281,6 +325,8 @@ github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hd
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-localereader v0.0.1 h1:ygSAOl7ZXTx4RdPYinUpg6W99U8jWvWi9Ye2JC/oIi4=
github.com/mattn/go-localereader v0.0.1/go.mod h1:8fBrzywKY7BI3czFoHkuzRoWE9C+EiG4R1k4Cjx5p88=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-runewidth v0.0.19 h1:v++JhqYnZuu5jSKrk9RbgF5v4CGUjqRfBm05byFGLdw=
github.com/mattn/go-runewidth v0.0.19/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
@@ -301,6 +347,8 @@ github.com/miekg/dns v1.1.68 h1:jsSRkNozw7G/mnmXULynzMNIsgY2dHC8LO6U6Ij2JEA=
github.com/miekg/dns v1.1.68/go.mod h1:fujopn7TB3Pu3JM69XaawiU0wqjpL9/8xGop5UrTPps=
github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc=
github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg=
github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4=
github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE=
github.com/moloch--/asciicast v0.1.1 h1:btwwniCK9br3ys33XZ71IG/M3ZujSSCcBzsH/sieiXg=
github.com/moloch--/asciicast v0.1.1/go.mod h1:OckO16UDLgxVLclrCnbocL1ix15Br/8Xv/caBoYq98o=
github.com/moloch--/go-keystone v0.0.2 h1:xThbYs45CSzoDUz5kBZxAyTPIpK3t5M7NB1x+NTTRL8=
@@ -309,6 +357,12 @@ github.com/moloch--/memmod v0.0.0-20230225130813-fd77d905589e h1:IkFCPlAa0iTiLxc
github.com/moloch--/memmod v0.0.0-20230225130813-fd77d905589e/go.mod h1:eYeI6cQ5YHhHt9i0BBW0zc1DaQnb4ZMXsSPuEV/V5Og=
github.com/moloch--/sgn v0.0.4 h1:90IC5rx3TmPEfrqPnENB0yyiqcbcNk9LqezXx7uruF0=
github.com/moloch--/sgn v0.0.4/go.mod h1:FyWfCtdynywgLmVR71vTJztZ3WkGSkR91Y4EOijUutE=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 h1:ZK8zHtRHOkbHy6Mmr5D264iyp3TiX5OmNcI5cIARiQI=
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6/go.mod h1:CJlz5H+gyd6CUWT45Oy4q24RdLyn7Md9Vj2/ldJBSIo=
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
github.com/muesli/termenv v0.16.0 h1:S5AlUN9dENB57rsbnkPyfdGuWIlkmzJjbFf0Tf5FWUc=
github.com/muesli/termenv v0.16.0/go.mod h1:ZRfOIKPFDYQoDFF4Olj7/QJbW60Ol/kL1pU3VfY/Cnk=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-sqlite3 v0.29.1 h1:NIi8AISWBToRHyoz01FXiTNvU147Tqdibgj2tFzJCqM=
@@ -416,13 +470,14 @@ github.com/u-root/uio v0.0.0-20240224005618-d2acac8f3701 h1:pyC9PaHYZFgEKFdlp3G8
github.com/u-root/uio v0.0.0-20240224005618-d2acac8f3701/go.mod h1:P3a5rG4X7tI17Nn3aOIAYr5HbIMukwXG0urG0WuL8OA=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/xlab/treeprint v1.2.0 h1:HzHnuAF1plUN2zGlAFHbSQP2qJ0ZAD3XF5XD7OesXRQ=
github.com/xlab/treeprint v1.2.0/go.mod h1:gj5Gd3gPdKtR1ikdDK6fnFLdmIS0X30kTTuNd/WEJu0=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yiya1989/sshkrb5 v0.0.1 h1:oAKpXLBeZExjYDV9eTL2KypmXU+5oPrJtqDuR5JbWic=
github.com/yiya1989/sshkrb5 v0.0.1/go.mod h1:r1bHcNa22x8rrg/Kj/cDF5bX6UtZrHiQgRFvj3Mnpl8=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -504,22 +559,20 @@ golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190530182044-ad28b68e88f1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201018230417-eeed37f84f13/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210217105451-b926d437f341/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220330033206-e17cdc41300f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220817070843-5a390386f1f2/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
@@ -645,5 +698,5 @@ mvdan.cc/sh/v3 v3.12.0 h1:ejKUR7ONP5bb+UGHGEG/k9V5+pRVIyD+LsZz7o8KHrI=
mvdan.cc/sh/v3 v3.12.0/go.mod h1:Se6Cj17eYSn+sNooLZiEUnNNmNxg0imoYlTu4CyaGyg=
software.sslmate.com/src/go-pkcs12 v0.4.0 h1:H2g08FrTvSFKUj+D309j1DPfk5APnIdAQAB8aEykJ5k=
software.sslmate.com/src/go-pkcs12 v0.4.0/go.mod h1:Qiz0EyvDRJjjxGyUQa2cCNZn/wMyzrRJ/qcDXOQazLI=
tailscale.com v1.88.4 h1:fXWotRMi9ZARyHRdKQa4ohXj8kqtemvvTzjreWLHVHo=
tailscale.com v1.88.4/go.mod h1:LHaTiwRgzebPDLgZ6RQQVzX+1SR5fbNl51fzm7UtMaw=
tailscale.com v1.90.6 h1:EhYPiZP/xcLeinikaLA0kn4CQT3+z9SZ13IB/kzWhd4=
tailscale.com v1.90.6/go.mod h1:+9EX6pOGCNa6pxCVRhhlJLy/qnkDzOplFYpeZyYlCT0=
@@ -37,17 +37,6 @@ package dnsclient
Stop = 4 bytes max
Size = 4 bytes max
Data = 78 - (2+4+4+4+4) ~= 60 bytes per query worst case
*** BASE58 ***
Base58 ~2% less efficient than Base64, but we can't use all 64 chars in DNS so
it's just not an option, we could potentially use some type of Base62 encoding
but those implementations are more complex and only marginally more efficient
than Base58, and Base58 avoids any complexities with '-' in domain names.
The idea is that since the server returns the messages CRC32 checksum we can detect
when the message is transparently corrupted by some rude resolver. So when we init
the session we send a few messages with random data to see if we can use Base58, and
fallback to Base32 if we detect problems.
*/
// {{if .Config.IncludeDNS}}
@@ -102,7 +91,6 @@ type DNSOptions struct {
RetryCount int
MaxErrors int
WorkersPerResolver int
ForceBase32 bool
NoTXT bool
ForceResolvConf string
ForceResolvers string
@@ -142,7 +130,6 @@ func ParseDNSOptions(c2URI *url.URL) *DNSOptions {
RetryCount: retryCount,
MaxErrors: maxErrors,
WorkersPerResolver: workersPerResolver,
ForceBase32: strings.ToLower(c2URI.Query().Get("force-base32")) == "true",
NoTXT: strings.ToLower(c2URI.Query().Get("notxt")) == "true",
ForceResolvConf: c2URI.Query().Get("force-resolv-conf"),
ForceResolvers: c2URI.Query().Get("resolvers"),
@@ -169,7 +156,6 @@ func NewDNSClient(parent string, opts *DNSOptions) *SliverDNSClient {
return &SliverDNSClient{
metadata: map[string]*ResolverMetadata{},
parent: parent,
forceBase32: opts.ForceBase32,
noTXT: opts.NoTXT,
forceResolvConf: opts.ForceResolvConf,
forceResolvers: opts.ForceResolvers,
@@ -181,7 +167,6 @@ func NewDNSClient(parent string, opts *DNSOptions) *SliverDNSClient {
WorkersPerResolver: opts.WorkersPerResolver,
subdataSpace: 254 - len(parent) - (1 + (254-len(parent))/64),
base32: encoders.Base32Encoder{},
base58: encoders.Base58Encoder{},
}
}
@@ -195,7 +180,6 @@ type SliverDNSClient struct {
retryWait time.Duration
retryCount int
queryTimeout time.Duration
forceBase32 bool
noTXT bool
forceResolvConf string
forceResolvers string
@@ -211,9 +195,6 @@ type SliverDNSClient struct {
WorkersPerResolver int
base32 encoders.Base32Encoder
base58 encoders.Base58Encoder
enableCaseSensitiveEncoder bool
}
// DNSWork - Single unit of work for DNSWorker
@@ -278,10 +259,9 @@ func (w *DNSWorker) Start(id int, recvQueue <-chan *DNSWork, sendQueue <-chan *D
// ResolverMetadata - Metadata for the resolver
type ResolverMetadata struct {
Address string
EnableBase58 bool
Metrics []time.Duration
Errors int
Address string
Metrics []time.Duration
Errors int
}
// SessionInit - Initialize DNS session
@@ -328,19 +308,13 @@ func (s *SliverDNSClient) SessionInit() error {
// {{end}}
return err
}
resolver, meta := s.randomResolver()
var encoder encoders.Encoder
if meta.EnableBase58 {
encoder = s.base58
} else {
encoder = s.base32
}
resolver, _ := s.randomResolver()
initMsg := &dnspb.DNSMessage{
ID: s.nextMsgID(),
Type: dnspb.DNSMessageType_INIT,
Size: uint32(len(initData)),
}
respData, err := s.sendInit(resolver, encoder, initMsg, initData)
respData, err := s.sendInit(resolver, s.base32, initMsg, initData)
if err != nil {
// {{if .Config.Debug}}
log.Printf("[dns] init msg send failure %v", err)
@@ -403,7 +377,8 @@ func (s *SliverDNSClient) sendInit(resolver DNSResolver, encoder encoders.Encode
return nil, err
}
resp := []byte{}
for _, subdata := range allSubdata {
receivedResponse := false
for index, subdata := range allSubdata {
var respData []byte
var err error
@@ -419,14 +394,25 @@ func (s *SliverDNSClient) sendInit(resolver DNSResolver, encoder encoders.Encode
return nil, err
}
if 0 < len(respData) {
resp = append(resp, respData...)
} else {
if len(allSubdata) == 1 {
resp = append(resp, respData...)
} else {
resp = respData
}
receivedResponse = true
} else if len(allSubdata) == 1 || (index == len(allSubdata)-1 && !receivedResponse) {
// {{if .Config.Debug}}
log.Printf("[dns] no data received in response")
// {{end}}
return nil, ErrInvalidResponse
}
}
if !receivedResponse {
// {{if .Config.Debug}}
log.Printf("[dns] no data received in response")
// {{end}}
return nil, ErrInvalidResponse
}
return resp, nil
}
@@ -460,8 +446,8 @@ func (s *SliverDNSClient) ReadEnvelope() (*pb.Envelope, error) {
log.Printf("[dns] read envelope ...")
// {{end}}
resolver, meta := s.randomResolver()
pollMsg, err := s.pollMsg(meta)
resolver, _ := s.randomResolver()
pollMsg, err := s.pollMsg()
if err != nil {
return nil, err
}
@@ -561,19 +547,13 @@ func (s *SliverDNSClient) CloseSession() error {
// parallelSend - send a full message to teh server
func (s *SliverDNSClient) parallelSend(data []byte) error {
var encoder encoders.Encoder
if s.enableCaseSensitiveEncoder {
encoder = s.base58
} else {
encoder = s.base32
}
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_DATA_FROM_IMPLANT,
ID: s.nextMsgID(),
Size: uint32(len(data)),
}
domains, err := s.SplitBuffer(msg, encoder, data)
domains, err := s.SplitBuffer(msg, s.base32, data)
if err != nil {
return err
}
@@ -889,7 +869,7 @@ func (s *SliverDNSClient) joinSubdataToParent(subdata string) (string, error) {
return domain, nil
}
func (s *SliverDNSClient) pollMsg(meta *ResolverMetadata) (string, error) {
func (s *SliverDNSClient) pollMsg() (string, error) {
nonceBuf := make([]byte, 8)
rand.Read(nonceBuf)
pollMsg, _ := proto.Marshal(&dnspb.DNSMessage{
@@ -897,13 +877,8 @@ func (s *SliverDNSClient) pollMsg(meta *ResolverMetadata) (string, error) {
Type: dnspb.DNSMessageType_POLL,
Data: nonceBuf,
})
if s.enableCaseSensitiveEncoder {
msg, _ := s.base58.Encode(pollMsg)
return string(msg), nil
} else {
msg, _ := s.base32.Encode(pollMsg)
return string(msg), nil
}
msg, _ := s.base32.Encode(pollMsg)
return string(msg), nil
}
func (s *SliverDNSClient) clearMsg(msgId uint32) (string, error) {
@@ -914,13 +889,8 @@ func (s *SliverDNSClient) clearMsg(msgId uint32) (string, error) {
Type: dnspb.DNSMessageType_CLEAR,
Data: nonceBuf,
})
if s.enableCaseSensitiveEncoder {
msg, _ := s.base58.Encode(clearMsg)
return string(msg), nil
} else {
msg, _ := s.base32.Encode(clearMsg)
return string(msg), nil
}
msg, _ := s.base32.Encode(clearMsg)
return string(msg), nil
}
func (s *SliverDNSClient) otpMsg() (string, error) {
@@ -936,7 +906,7 @@ func (s *SliverDNSClient) otpMsg() (string, error) {
return string(msg), nil
}
// fingerprintResolver - Fingerprints resolve to determine if we can use a case sensitive encoding
// fingerprintResolver - Fingerprints resolve to validate base32 transport.
func (s *SliverDNSClient) fingerprintResolvers() {
wg := &sync.WaitGroup{}
// {{if .Config.Debug}}
@@ -960,15 +930,14 @@ func (s *SliverDNSClient) fingerprintResolvers() {
// {{if .Config.Debug}}
for _, result := range s.metadata {
log.Printf("[dns] %s: avg rtt %s, base58: %v, errors %d",
result.Address, s.averageRtt(result), result.EnableBase58, result.Errors)
log.Printf("[dns] %s: avg rtt %s, errors %d",
result.Address, s.averageRtt(result), result.Errors)
}
// {{end}}
// NOTE: In the future we may want to add a configurable error threshold for now
// if we encounter any errors we don't use the resolver.
workingResolvers := []DNSResolver{}
allSupportBase58 := true
for _, resolver := range s.resolvers {
meta := s.metadata[resolver.Address()]
if 0 < meta.Errors {
@@ -977,41 +946,24 @@ func (s *SliverDNSClient) fingerprintResolvers() {
// {{end}}
continue
}
if !meta.EnableBase58 {
allSupportBase58 = false
}
workingResolvers = append(workingResolvers, resolver)
}
if allSupportBase58 && !s.forceBase32 {
s.enableCaseSensitiveEncoder = true
}
s.resolvers = workingResolvers
}
// Fingerprints a single resolver to determine if we can use a case sensitive encoding, average
// round trip time, and if it works at all
// Fingerprints a single resolver to measure RTT and validate base32 transport.
func (s *SliverDNSClient) fingerprintResolver(id int, wg *sync.WaitGroup, results chan<- *ResolverMetadata, resolver DNSResolver) {
defer wg.Done()
meta := &ResolverMetadata{
Address: resolver.Address(),
EnableBase58: false,
Metrics: []time.Duration{},
Errors: 0,
}
s.benchmark(id, s.base32, resolver, meta)
if meta.Errors == 0 && !s.forceBase32 {
s.benchmark(id, s.base58, resolver, meta)
if meta.Errors == 0 {
meta.EnableBase58 = true
} else {
meta.EnableBase58 = false
meta.Errors = 0 // Reset base32 error count
}
Address: resolver.Address(),
Metrics: []time.Duration{},
Errors: 0,
}
s.benchmark(id, resolver, meta)
results <- meta
}
func (s *SliverDNSClient) benchmark(id int, encoder encoders.Encoder, resolver DNSResolver, meta *ResolverMetadata) {
func (s *SliverDNSClient) benchmark(id int, resolver DNSResolver, meta *ResolverMetadata) {
for index := 0; index < metricsMaxSize/2; index++ {
finger, fingerChecksum, err := s.fingerprintMsg(id)
if err != nil {
@@ -1021,7 +973,7 @@ func (s *SliverDNSClient) benchmark(id int, encoder encoders.Encoder, resolver D
// {{end}}
continue
}
encodedValue, _ := encoder.Encode(finger)
encodedValue, _ := s.base32.Encode(finger)
domain, err := s.joinSubdataToParent(string(encodedValue))
if err != nil {
meta.Errors++
@@ -36,9 +36,10 @@ import (
const (
// Do not change these without updating the tests
parent1 = ".1.example.com."
parent2 = ".something-longer.example.com."
parent3 = ".something-even-longer.example.computer."
parent1 = ".1.example.com."
parent2 = ".something-longer.example.com."
parent3 = ".something-even-longer.example.computer."
base32Alphabet = "ab1c2d3e4f5g6h7j8k9m0npqrtuvwxyz"
)
var (
@@ -70,35 +71,39 @@ func randomData(size int) []byte {
return buf
}
func TestSplitBufferBase58(t *testing.T) {
t.Logf("Testing with client1")
client1 := NewDNSClient(parent1, opts)
testData := randomData(2048)
clientSplitBuffer(t, client1, encoders.Base58{}, testData)
t.Logf("Testing with client2")
client2 := NewDNSClient(parent2, opts)
testData2 := randomData(2048)
clientSplitBuffer(t, client2, encoders.Base58{}, testData2)
t.Logf("Testing with client3")
client3 := NewDNSClient(parent3, opts)
testData3 := randomData(2048)
clientSplitBuffer(t, client3, encoders.Base58{}, testData3)
t.Logf("Testing with client max")
clientMax := NewDNSClient(parentMax, opts)
testDataMax := randomData(2048)
clientSplitBuffer(t, clientMax, encoders.Base58{}, testDataMax)
t.Logf("Testing all clients with randomly sized data")
for _, client := range []*SliverDNSClient{client1, client2, client3} {
for count := 0; count < 10; count++ {
testData := randomDataRandomSize(2 * 1024 * 1024)
clientSplitBuffer(t, client, encoders.Base58{}, testData)
func isBase32String(value string) bool {
for i := 0; i < len(value); i++ {
if strings.IndexByte(base32Alphabet, value[i]) == -1 {
return false
}
}
return true
}
type initSequenceResolver struct {
responses [][]byte
index int
}
func (r *initSequenceResolver) Address() string {
return "init-sequence"
}
func (r *initSequenceResolver) A(domain string) ([]byte, time.Duration, error) {
return nil, 0, nil
}
func (r *initSequenceResolver) AAAA(domain string) ([]byte, time.Duration, error) {
return nil, 0, nil
}
func (r *initSequenceResolver) TXT(domain string) ([]byte, time.Duration, error) {
if r.index >= len(r.responses) {
return nil, 0, nil
}
resp := r.responses[r.index]
r.index++
return resp, 0, nil
}
func TestSplitBufferBase32(t *testing.T) {
@@ -132,6 +137,69 @@ func TestSplitBufferBase32(t *testing.T) {
}
}
func TestSplitBufferDomainConstraints(t *testing.T) {
client := NewDNSClient(parent1, opts)
testData := randomData(8192)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_DATA_FROM_IMPLANT,
Size: uint32(len(testData)),
}
domains, err := client.SplitBuffer(msg, encoders.Base32{}, testData)
if err != nil {
t.Fatalf("Unexpected error splitting buffer: %s", err)
}
for _, domain := range domains {
if !strings.HasSuffix(domain, parent1) {
t.Fatalf("Domain missing parent suffix: %s", domain)
}
if len(domain) > 254 {
t.Fatalf("Domain exceeds DNS length limit: %d", len(domain))
}
subdata := strings.TrimSuffix(domain, parent1)
if !isBase32String(strings.ReplaceAll(subdata, ".", "")) {
t.Fatalf("Subdata contains non-base32 characters: %s", subdata)
}
for _, label := range strings.Split(subdata, ".") {
if len(label) > 63 {
t.Fatalf("Label exceeds 63 chars: %s", label)
}
}
}
}
func TestSendInitAllowsEmptyResponses(t *testing.T) {
client := NewDNSClient(parent1, opts)
testData := randomData(4096)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_INIT,
ID: 1,
Size: uint32(len(testData)),
}
domains, err := client.SplitBuffer(msg, encoders.Base32{}, testData)
if err != nil {
t.Fatalf("Unexpected error splitting buffer: %s", err)
}
if len(domains) < 2 {
t.Fatalf("expected multiple subdata domains, got %d", len(domains))
}
expected := []byte{1, 2, 3, 4}
responses := make([][]byte, len(domains))
responses[len(responses)-1] = expected
resolver := &initSequenceResolver{responses: responses}
resp, err := client.sendInit(resolver, encoders.Base32{}, msg, testData)
if err != nil {
t.Fatalf("sendInit failed: %s", err)
}
if !bytes.Equal(resp, expected) {
t.Fatalf("unexpected init response: %v", resp)
}
if resolver.index != len(domains) {
t.Fatalf("expected %d resolver calls, got %d", len(domains), resolver.index)
}
}
func clientSplitBuffer(t *testing.T, client *SliverDNSClient, encoder encoders.Encoder, testData []byte) {
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_DATA_FROM_IMPLANT,
@@ -162,25 +230,74 @@ func clientSplitBuffer(t *testing.T, client *SliverDNSClient, encoder encoders.E
}
}
// func addTestResolver(client *SliverDNSClient, enableBase58 bool) {
// client.resolvers = []DNSResolver{
// &GenericResolver{
// address: "127.0.0.1:53",
// retries: 1,
// retryWait: retryWait,
// resolver: &dns.Client{
// ReadTimeout: timeout,
// WriteTimeout: timeout,
// },
// base64: encoders.Base64{},
// },
// }
// client.metadata["127.0.0.1:53"] = &ResolverMetadata{
// Address: "127.0.0.1:53",
// EnableBase58: enableBase58,
// Errors: 0,
// }
// }
func TestOTPMsgRoundTrip(t *testing.T) {
client := NewDNSClient(parent1, opts)
encoded, err := client.otpMsg()
if err != nil {
t.Fatalf("Unexpected otpMsg error: %s", err)
}
if !isBase32String(encoded) {
t.Fatalf("otpMsg contains non-base32 characters: %s", encoded)
}
raw, err := encoders.Base32{}.Decode([]byte(encoded))
if err != nil {
t.Fatalf("Unexpected otpMsg decode error: %s", err)
}
msg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(raw, msg); err != nil {
t.Fatalf("Unexpected otpMsg unmarshal error: %s", err)
}
if msg.Type != dnspb.DNSMessageType_TOTP {
t.Fatalf("Unexpected otpMsg type: %v", msg.Type)
}
if msg.ID != 0 {
t.Fatalf("Unexpected otpMsg id: %d", msg.ID)
}
}
func TestPollAndClearMsgRoundTrip(t *testing.T) {
client := NewDNSClient(parent1, opts)
client.dnsSessionID = 42
pollEncoded, err := client.pollMsg()
if err != nil {
t.Fatalf("Unexpected pollMsg error: %s", err)
}
clearEncoded, err := client.clearMsg(99)
if err != nil {
t.Fatalf("Unexpected clearMsg error: %s", err)
}
for _, test := range []struct {
name string
encoded string
wantType dnspb.DNSMessageType
wantID uint32
wantDataSz int
}{
{name: "poll", encoded: pollEncoded, wantType: dnspb.DNSMessageType_POLL, wantID: 42, wantDataSz: 8},
{name: "clear", encoded: clearEncoded, wantType: dnspb.DNSMessageType_CLEAR, wantID: 99, wantDataSz: 8},
} {
if !isBase32String(test.encoded) {
t.Fatalf("%s contains non-base32 characters: %s", test.name, test.encoded)
}
raw, err := encoders.Base32{}.Decode([]byte(test.encoded))
if err != nil {
t.Fatalf("%s decode error: %s", test.name, err)
}
msg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(raw, msg); err != nil {
t.Fatalf("%s unmarshal error: %s", test.name, err)
}
if msg.Type != test.wantType {
t.Fatalf("%s type mismatch: %v", test.name, msg.Type)
}
if msg.ID != test.wantID {
t.Fatalf("%s id mismatch: %d", test.name, msg.ID)
}
if len(msg.Data) != test.wantDataSz {
t.Fatalf("%s data length mismatch: %d", test.name, len(msg.Data))
}
}
}
func TestSubdataSpace(t *testing.T) {
+1 -13
View File
@@ -252,19 +252,7 @@ func setupGo(appDir string) error {
return err
}
return setupSGN(appDir)
}
func setupSGN(appDir string) error {
goBinPath := filepath.Join(appDir, "go", "bin")
sgnZipFSPath := path.Join("fs", runtime.GOOS, runtime.GOARCH, "sgn.zip")
sgnZip, err := assetsFs.ReadFile(sgnZipFSPath)
if err != nil {
setupLog.Errorf("static asset not found: %s", sgnZipFSPath)
return err
}
_, err = unzipBuf(sgnZip, goBinPath)
return err
return nil
}
// SetupGoPath - Extracts dependencies to goPathSrc
+58 -25
View File
@@ -45,7 +45,6 @@ import (
"strings"
"sync"
"time"
"unicode"
consts "github.com/bishopfox/sliver/client/constants"
"github.com/bishopfox/sliver/protobuf/dnspb"
@@ -419,32 +418,17 @@ func (s *SliverDNSServer) handleC2(domain string, req *dns.Msg) *dns.Msg {
// checksum calculation here to ensure that no one accidentally calculates the crc32
// of the plaintext data (that would be very bad).
func (s *SliverDNSServer) decodeSubdata(subdomain string) (*dnspb.DNSMessage, uint32, error) {
subdata := strings.Join(strings.Split(subdomain, "."), "")
subdata := strings.ToLower(strings.Join(strings.Split(subdomain, "."), ""))
dnsLog.Debugf("subdata = %s", subdata)
encoders := s.determineLikelyEncoders(subdata)
for _, encoder := range encoders {
data, err := encoder.Decode([]byte(subdata))
if err == nil {
msg := &dnspb.DNSMessage{}
err = proto.Unmarshal(data, msg)
if err == nil {
return msg, crc32.ChecksumIEEE(data), nil
}
}
dnsLog.Debugf("failed to decode subdata with %#v (%s)", encoder, err)
data, err := encoders.Base32{}.Decode([]byte(subdata))
if err != nil {
return nil, 0, ErrInvalidMsg
}
return nil, 0, ErrInvalidMsg
}
// Returns the most likely -> least likely encoders, if decoding fails fallback to
// the next encoder until we run out of options.
func (s *SliverDNSServer) determineLikelyEncoders(subdata string) []encoders.Encoder {
for _, char := range subdata {
if unicode.IsUpper(char) {
return []encoders.Encoder{encoders.Base58{}, encoders.Base32{}}
}
msg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(data, msg); err != nil {
return nil, 0, ErrInvalidMsg
}
return []encoders.Encoder{encoders.Base32{}, encoders.Base58{}}
return msg, crc32.ChecksumIEEE(data), nil
}
func (s *SliverDNSServer) nameErrorResp(req *dns.Msg) *dns.Msg {
@@ -454,6 +438,13 @@ func (s *SliverDNSServer) nameErrorResp(req *dns.Msg) *dns.Msg {
return resp
}
func (s *SliverDNSServer) emptySuccessResp(req *dns.Msg) *dns.Msg {
resp := new(dns.Msg)
resp.SetReply(req)
resp.Authoritative = true
return resp
}
func (s *SliverDNSServer) refusedErrorResp(req *dns.Msg) *dns.Msg {
resp := new(dns.Msg)
resp.SetRcode(req, dns.RcodeRefused)
@@ -461,6 +452,38 @@ func (s *SliverDNSServer) refusedErrorResp(req *dns.Msg) *dns.Msg {
return resp
}
func (s *SliverDNSServer) accumulateInitData(msg *dnspb.DNSMessage) ([]byte, bool, error) {
if msg.Size == 0 {
return nil, false, ErrInvalidMsg
}
pendingValue, loaded := s.messages.Load(msg.ID)
if !loaded {
pendingValue = &PendingEnvelope{
Size: msg.Size,
received: uint32(0),
messages: map[uint32][]byte{},
mutex: &sync.Mutex{},
complete: false,
}
actual, _ := s.messages.LoadOrStore(msg.ID, pendingValue)
pendingValue = actual
}
pending := pendingValue.(*PendingEnvelope)
if pending.Size != msg.Size && msg.Size != 0 {
s.messages.Delete(msg.ID)
return nil, false, ErrInvalidMsg
}
if !pending.Insert(msg) {
return nil, false, nil
}
data, err := pending.Reassemble()
s.messages.Delete(msg.ID)
if err != nil {
return nil, false, err
}
return data, true, nil
}
// ---------------------------
// DNS Message Handlers
// ---------------------------
@@ -502,7 +525,7 @@ func (s *SliverDNSServer) handleDNSSessionInit(domain string, msg *dnspb.DNSMess
dnsLog.Debugf("[session init] with dns session id %d", msg.ID&sessionIDBitMask)
loadSession, _ := s.sessions.Load(msg.ID & sessionIDBitMask)
dnsSession := loadSession.(*DNSSession)
if len(msg.Data) <= 32 {
if msg.Size <= 32 {
dnsLog.Warnf("[session init] invalid msg data length")
return s.refusedErrorResp(req)
}
@@ -511,6 +534,16 @@ func (s *SliverDNSServer) handleDNSSessionInit(domain string, msg *dnspb.DNSMess
return s.refusedErrorResp(req)
}
initData, complete, err := s.accumulateInitData(msg)
if err != nil {
dnsLog.Errorf("[session init] error reassembling init data: %s", err)
return s.refusedErrorResp(req)
}
if !complete {
return s.emptySuccessResp(req)
}
msg.Data = initData
var publicKeyDigest [32]byte
copy(publicKeyDigest[:], msg.Data[:32])
implantBuild, err := db.ImplantBuildByPublicKeyDigest(publicKeyDigest)
+696 -9
View File
@@ -21,16 +21,24 @@ package c2
import (
"bytes"
"crypto/rand"
"encoding/binary"
"fmt"
"hash/crc32"
insecureRand "math/rand"
"strings"
"sync"
"testing"
"time"
implantCrypto "github.com/bishopfox/sliver/implant/sliver/cryptography"
implantEncoders "github.com/bishopfox/sliver/implant/sliver/encoders"
"github.com/bishopfox/sliver/implant/sliver/transports/dnsclient"
"github.com/bishopfox/sliver/protobuf/dnspb"
"github.com/bishopfox/sliver/protobuf/sliverpb"
"github.com/bishopfox/sliver/server/core"
serverCrypto "github.com/bishopfox/sliver/server/cryptography"
"github.com/bishopfox/sliver/util/encoders"
"github.com/miekg/dns"
"google.golang.org/protobuf/proto"
)
@@ -61,6 +69,93 @@ func shuffleDNSMsgs(a []*dnspb.DNSMessage) {
}
}
type testDNSResolver struct {
server *SliverDNSServer
parent string
base64 implantEncoders.Base64Encoder
}
func (r *testDNSResolver) Address() string {
return "test-resolver"
}
func (r *testDNSResolver) A(domain string) ([]byte, time.Duration, error) {
resp, err := r.exchange(domain, dns.TypeA)
if err != nil {
return nil, 0, err
}
records := []byte{}
for _, answer := range resp.Answer {
if a, ok := answer.(*dns.A); ok {
records = append(records, []byte(a.A)...)
}
}
return records, 0, nil
}
func (r *testDNSResolver) AAAA(domain string) ([]byte, time.Duration, error) {
resp, err := r.exchange(domain, dns.TypeAAAA)
if err != nil {
return nil, 0, err
}
records := make([]byte, 512)
dataSize := uint32(0)
for _, answer := range resp.Answer {
if aaaa, ok := answer.(*dns.AAAA); ok {
chunkMeta := uint32(aaaa.Hdr.Ttl)
chunkIdx := (chunkMeta & 0xff00) >> 8
tempSize := chunkMeta & 0xff
if dataSize != 0 && tempSize != dataSize {
return nil, 0, dnsclient.ErrInvalidResponse
}
if dataSize == 0 {
dataSize = tempSize
}
copy(records[chunkIdx*16:], []byte(aaaa.AAAA))
}
}
if dataSize == 0 {
return nil, 0, nil
}
data := make([]byte, dataSize)
copy(data, records[:dataSize])
return data, 0, nil
}
func (r *testDNSResolver) TXT(domain string) ([]byte, time.Duration, error) {
resp, err := r.exchange(domain, dns.TypeTXT)
if err != nil {
return nil, 0, err
}
records := ""
for _, answer := range resp.Answer {
if txt, ok := answer.(*dns.TXT); ok {
records += strings.Join(txt.Txt, "")
}
}
if records == "" {
return nil, 0, nil
}
data, err := r.base64.Decode([]byte(records))
if err != nil {
return nil, 0, err
}
return data, 0, nil
}
func (r *testDNSResolver) exchange(domain string, qtype uint16) (*dns.Msg, error) {
req := new(dns.Msg)
req.SetQuestion(domain, qtype)
resp := r.server.handleC2(r.parent, req)
if resp == nil {
return nil, dnsclient.ErrInvalidResponse
}
if resp.Rcode != dns.RcodeSuccess {
return resp, dnsclient.ErrInvalidResponse
}
return resp, nil
}
func randomDNSMsgs(t *testing.T, parent string, maxSize int, encoder encoders.Encoder, client *dnsclient.SliverDNSClient) ([]*dnspb.DNSMessage, []byte) {
testData := randomDataRandomSize(maxSize)
dnsMsgs := []*dnspb.DNSMessage{}
@@ -93,12 +188,10 @@ func randomDNSMsgs(t *testing.T, parent string, maxSize int, encoder encoders.En
func TestPendingEnvelopes(t *testing.T) {
// *** Small ***
for i := 0; i < 100; i++ {
reassemble(t, example1, 256, encoders.Base58{})
reassemble(t, example1, 256, encoders.Base32{})
}
// *** Large ***
for i := 0; i < 100; i++ {
reassemble(t, example1, 30*1024, encoders.Base58{})
reassemble(t, example1, 30*1024, encoders.Base32{})
}
}
@@ -139,6 +232,194 @@ func reassemble(t *testing.T, parent string, size int, encoder encoders.Encoder)
}
}
func TestDNSHelloRoundTrip(t *testing.T) {
server := newTestDNSServer()
resolver := &testDNSResolver{server: server, parent: example1}
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_TOTP,
ID: 0,
}
domain, err := encodeMessageDomain(example1, msg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
data, _, err := resolver.A(domain)
if err != nil {
t.Fatalf("resolver.A failed: %s", err)
}
if len(data) != 4 {
t.Fatalf("expected 4 bytes in response, got %d", len(data))
}
sessionID := binary.LittleEndian.Uint32(data) & sessionIDBitMask
if sessionID == 0 {
t.Fatal("expected non-zero session id")
}
if _, ok := server.sessions.Load(sessionID); !ok {
t.Fatalf("expected session %d to be stored", sessionID)
}
}
func TestDNSNOPRoundTripChecksum(t *testing.T) {
server := newTestDNSServer()
sessionID := uint32(0x123456)
dnsSession := newTestDNSSession(sessionID)
server.sessions.Store(sessionID, dnsSession)
payload := make([]byte, 16)
rand.Read(payload)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_NOP,
ID: (1 << 24) | sessionID,
Data: payload,
}
raw, err := proto.Marshal(msg)
if err != nil {
t.Fatalf("marshal failed: %s", err)
}
domain, err := encodeMessageDomain(example1, msg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
resolver := &testDNSResolver{server: server, parent: example1}
data, _, err := resolver.A(domain)
if err != nil {
t.Fatalf("resolver.A failed: %s", err)
}
if len(data) != 4 {
t.Fatalf("expected 4 bytes in response, got %d", len(data))
}
got := binary.LittleEndian.Uint32(data)
want := crc32.ChecksumIEEE(raw)
if got != want {
t.Fatalf("checksum mismatch: got %d want %d", got, want)
}
}
func TestDNSPollManifestReuse(t *testing.T) {
server := newTestDNSServer()
sessionID := uint32(0x654321)
dnsSession := newTestDNSSession(sessionID)
server.sessions.Store(sessionID, dnsSession)
resolver := &testDNSResolver{server: server, parent: example1}
pollMsg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_POLL,
ID: sessionID,
Data: []byte{1, 2, 3, 4, 5, 6, 7, 8},
}
pollDomain, err := encodeMessageDomain(example1, pollMsg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
respData, _, err := resolver.TXT(pollDomain)
if err != nil {
t.Fatalf("resolver.TXT failed: %s", err)
}
manifest := &dnspb.DNSMessage{}
if err := proto.Unmarshal(respData, manifest); err != nil {
t.Fatalf("manifest unmarshal failed: %s", err)
}
if manifest.Type != dnspb.DNSMessageType_MANIFEST || manifest.Size != 0 || manifest.ID != 0 {
t.Fatalf("unexpected empty manifest: %+v", manifest)
}
msgID := uint32(0x01000000 | sessionID)
dnsSession.outgoingMsgIDs = append(dnsSession.outgoingMsgIDs, msgID)
dnsSession.outgoingBuffers[msgID] = bytes.Repeat([]byte{0xAB}, 32)
respData, _, err = resolver.TXT(pollDomain)
if err != nil {
t.Fatalf("resolver.TXT failed: %s", err)
}
manifest = &dnspb.DNSMessage{}
if err := proto.Unmarshal(respData, manifest); err != nil {
t.Fatalf("manifest unmarshal failed: %s", err)
}
if manifest.Type != dnspb.DNSMessageType_MANIFEST || manifest.ID != msgID || manifest.Size != 32 {
t.Fatalf("unexpected manifest: %+v", manifest)
}
if dnsSession.dnsIdMsgIdMap[pollMsg.ID] != msgID {
t.Fatalf("expected dnsIdMsgIdMap to track msg id %d", msgID)
}
respData, _, err = resolver.TXT(pollDomain)
if err != nil {
t.Fatalf("resolver.TXT failed: %s", err)
}
manifest = &dnspb.DNSMessage{}
if err := proto.Unmarshal(respData, manifest); err != nil {
t.Fatalf("manifest unmarshal failed: %s", err)
}
if manifest.Type != dnspb.DNSMessageType_MANIFEST || manifest.ID != msgID || manifest.Size != 32 {
t.Fatalf("unexpected reused manifest: %+v", manifest)
}
}
func TestDNSReadEnvelopeRoundTripTXT(t *testing.T) {
runDNSReadEnvelopeRoundTrip(t, false)
}
func TestDNSReadEnvelopeRoundTripAAAA(t *testing.T) {
runDNSReadEnvelopeRoundTrip(t, true)
}
func TestDNSWriteEnvelopeRoundTrip(t *testing.T) {
server := newTestDNSServer()
sessionID := uint32(0x102030)
key := implantCrypto.RandomSymmetricKey()
dnsSession := newTestDNSSession(sessionID)
dnsSession.CipherCtx = serverCrypto.NewCipherContext(key)
dnsSession.ImplantConn = core.NewImplantConnection("dns", "n/a")
respCh := make(chan *sliverpb.Envelope, 1)
dnsSession.ImplantConn.Resp[77] = respCh
server.sessions.Store(sessionID, dnsSession)
resolver := &testDNSResolver{server: server, parent: example1}
envelope := &sliverpb.Envelope{
ID: 77,
Type: sliverpb.MsgPing,
Data: []byte("hello"),
}
sendEnvelopeViaDNS(t, resolver, sessionID, key, envelope)
select {
case got := <-respCh:
if !proto.Equal(got, envelope) {
t.Fatalf("envelope mismatch: %#v != %#v", got, envelope)
}
case <-time.After(2 * time.Second):
t.Fatal("timeout waiting for envelope")
}
}
func TestDNSDataToImplantRejectsInvalidRead(t *testing.T) {
server := newTestDNSServer()
sessionID := uint32(0x121212)
dnsSession := newTestDNSSession(sessionID)
dnsSession.outgoingBuffers[0x01000000|sessionID] = bytes.Repeat([]byte{0xCD}, 16)
server.sessions.Store(sessionID, dnsSession)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_DATA_TO_IMPLANT,
ID: 0x01000000 | sessionID,
Start: 5,
Stop: 4,
}
domain, err := encodeMessageDomain(example1, msg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
req := new(dns.Msg)
req.SetQuestion(domain, dns.TypeTXT)
resp := server.handleC2(example1, req)
if resp == nil || resp.Rcode != dns.RcodeRefused {
t.Fatalf("expected refused response, got %+v", resp)
}
}
func TestIsC2Domain(t *testing.T) {
listener := StartDNSListener("", uint16(9999), c2Domains, false, true)
isC2, domain := listener.isC2SubDomain(c2Domains, "asdf.1.example.com.")
@@ -162,14 +443,420 @@ func TestIsC2Domain(t *testing.T) {
}
}
func TestDetermineLikelyEncoders(t *testing.T) {
func TestDecodeSubdataBase32(t *testing.T) {
listener := StartDNSListener("", uint16(9999), c2Domains, false, true)
sample := randomDataRandomSize(2048)
encodedSample, _ := encoders.Base58{}.Encode(sample)
b58Sample := string(encodedSample)
likelyEncoders := listener.determineLikelyEncoders(b58Sample)
_, err := likelyEncoders[0].Decode([]byte(b58Sample))
payload := make([]byte, 64)
rand.Read(payload)
original := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_NOP,
ID: 42,
Data: payload,
}
raw, err := proto.Marshal(original)
if err != nil {
t.Error("DetermineLikelyEncoders failed to decode sample")
t.Fatalf("failed to marshal message: %s", err)
}
encoded, _ := encoders.Base32{}.Encode(raw)
if len(encoded) < 12 {
t.Fatalf("encoded sample too short: %d", len(encoded))
}
subdomain := fmt.Sprintf("%s.%s", string(encoded[:12]), string(encoded[12:]))
for _, sample := range []string{subdomain, strings.ToUpper(subdomain)} {
msg, checksum, err := listener.decodeSubdata(sample)
if err != nil {
t.Fatalf("decodeSubdata failed: %s", err)
}
if !proto.Equal(msg, original) {
t.Fatalf("decoded message mismatch: %#v != %#v", msg, original)
}
if checksum != crc32.ChecksumIEEE(raw) {
t.Fatalf("unexpected checksum: %d", checksum)
}
}
}
func TestDecodeSubdataRejectsInvalid(t *testing.T) {
listener := StartDNSListener("", uint16(9999), c2Domains, false, true)
_, _, err := listener.decodeSubdata("invalid")
if err != ErrInvalidMsg {
t.Fatalf("expected ErrInvalidMsg, got %v", err)
}
}
func TestDecodeSubdataRejectsInvalidProto(t *testing.T) {
listener := StartDNSListener("", uint16(9999), c2Domains, false, true)
encoded, err := implantEncoders.Base32Encoder{}.Encode([]byte{0x00})
if err != nil {
t.Fatalf("encode failed: %s", err)
}
_, _, err = listener.decodeSubdata(string(encoded))
if err != ErrInvalidMsg {
t.Fatalf("expected ErrInvalidMsg, got %v", err)
}
}
func TestHandleC2RejectsUnknownSession(t *testing.T) {
server := newTestDNSServer()
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_NOP,
ID: 0x00123456,
Data: []byte{1, 2, 3},
}
domain, err := encodeMessageDomain(example1, msg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
req := new(dns.Msg)
req.SetQuestion(domain, dns.TypeA)
resp := server.handleC2(example1, req)
if resp == nil || resp.Rcode != dns.RcodeNameError {
t.Fatalf("expected name error response, got %+v", resp)
}
}
func TestAccumulateInitDataReassembles(t *testing.T) {
server := newTestDNSServer()
client := dnsclient.NewDNSClient(example1, opts)
testData := make([]byte, 2048)
rand.Read(testData)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_INIT,
ID: 0x01020304,
Size: uint32(len(testData)),
}
domains, err := client.SplitBuffer(msg, implantEncoders.Base32Encoder{}, testData)
if err != nil {
t.Fatalf("SplitBuffer failed: %s", err)
}
if len(domains) < 2 {
t.Fatalf("expected multiple init submessages, got %d", len(domains))
}
subMsgs := make([]*dnspb.DNSMessage, 0, len(domains))
for _, domain := range domains {
subdata := strings.TrimSuffix(domain, example1)
subdata = strings.ReplaceAll(subdata, ".", "")
raw, err := implantEncoders.Base32Encoder{}.Decode([]byte(subdata))
if err != nil {
t.Fatalf("decode subdata failed: %s", err)
}
subMsg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(raw, subMsg); err != nil {
t.Fatalf("unmarshal submsg failed: %s", err)
}
subMsgs = append(subMsgs, subMsg)
}
shuffleDNSMsgs(subMsgs)
var reassembled []byte
for _, subMsg := range subMsgs {
data, complete, err := server.accumulateInitData(subMsg)
if err != nil {
t.Fatalf("accumulateInitData failed: %s", err)
}
if !complete && data != nil {
t.Fatalf("unexpected data before completion")
}
if complete {
reassembled = data
}
}
if reassembled == nil {
t.Fatalf("expected reassembled init data")
}
if !bytes.Equal(reassembled, testData) {
t.Fatalf("reassembled init data mismatch")
}
}
func TestSplitToChunksPadding(t *testing.T) {
data := []byte{1, 2, 3, 4, 5, 6}
chunks := splitToChunks(data, 4)
if len(chunks) != 2 {
t.Fatalf("expected 2 chunks, got %d", len(chunks))
}
if !bytes.Equal(chunks[0], []byte{1, 2, 3, 4}) {
t.Fatalf("unexpected first chunk: %v", chunks[0])
}
if !bytes.Equal(chunks[1], []byte{5, 6, 0, 0}) {
t.Fatalf("unexpected second chunk: %v", chunks[1])
}
for _, chunk := range chunks {
if len(chunk) != 4 {
t.Fatalf("unexpected chunk length: %d", len(chunk))
}
}
}
func TestOutgoingReadBoundaries(t *testing.T) {
session := &DNSSession{
outgoingBuffers: map[uint32][]byte{
1: {1, 2, 3, 4},
},
outgoingMutex: &sync.RWMutex{},
}
if _, err := session.OutgoingRead(1, 0, 0); err == nil {
t.Fatal("expected error for stop <= start")
}
if _, err := session.OutgoingRead(1, 0, 5); err == nil {
t.Fatal("expected error for stop > length")
}
if _, err := session.OutgoingRead(1, 5, 6); err == nil {
t.Fatal("expected error for start > length")
}
if _, err := session.OutgoingRead(2, 0, 1); err == nil {
t.Fatal("expected error for missing buffer")
}
read, err := session.OutgoingRead(1, 1, 3)
if err != nil {
t.Fatalf("unexpected error for valid read: %s", err)
}
if !bytes.Equal(read, []byte{2, 3}) {
t.Fatalf("unexpected read data: %v", read)
}
}
func runDNSReadEnvelopeRoundTrip(t *testing.T, noTXT bool) {
t.Helper()
server := newTestDNSServer()
sessionID := uint32(0x2468AC)
key := implantCrypto.RandomSymmetricKey()
dnsSession := newTestDNSSession(sessionID)
dnsSession.CipherCtx = serverCrypto.NewCipherContext(key)
server.sessions.Store(sessionID, dnsSession)
envelope := &sliverpb.Envelope{
ID: 99,
Type: sliverpb.MsgPing,
Data: []byte("roundtrip"),
}
if err := dnsSession.StageOutgoingEnvelope(envelope); err != nil {
t.Fatalf("StageOutgoingEnvelope failed: %s", err)
}
if len(dnsSession.outgoingMsgIDs) != 1 {
t.Fatalf("expected 1 outgoing message id, got %d", len(dnsSession.outgoingMsgIDs))
}
msgID := dnsSession.outgoingMsgIDs[0]
resolver := &testDNSResolver{server: server, parent: example1}
got, err := readEnvelopeViaDNS(t, resolver, sessionID, key, noTXT)
if err != nil {
t.Fatalf("ReadEnvelope failed: %s", err)
}
if got == nil || !proto.Equal(got, envelope) {
t.Fatalf("envelope mismatch: %#v != %#v", got, envelope)
}
if _, ok := dnsSession.outgoingBuffers[msgID]; ok {
t.Fatalf("expected outgoing buffer %d to be cleared", msgID)
}
}
func newTestDNSServer() *SliverDNSServer {
return &SliverDNSServer{
sessions: &sync.Map{},
messages: &sync.Map{},
TTL: 0,
MaxTXTLength: defaultMaxTXTLength,
EnforceOTP: false,
}
}
func newTestDNSSession(sessionID uint32) *DNSSession {
return &DNSSession{
ID: sessionID,
dnsIdMsgIdMap: map[uint32]uint32{},
outgoingMsgIDs: []uint32{},
outgoingBuffers: map[uint32][]byte{},
outgoingMutex: &sync.RWMutex{},
incomingEnvelopes: map[uint32]*PendingEnvelope{},
incomingMutex: &sync.Mutex{},
msgCount: 0,
}
}
func encodeMessageDomain(parent string, msg *dnspb.DNSMessage) (string, error) {
raw, err := proto.Marshal(msg)
if err != nil {
return "", err
}
encoded, err := implantEncoders.Base32Encoder{}.Encode(raw)
if err != nil {
return "", err
}
return joinSubdataToParent(string(encoded), parent), nil
}
func joinSubdataToParent(subdata string, parent string) string {
subdomains := []string{}
for i := 0; i < len(subdata); i += 63 {
stop := i + 63
if len(subdata) < stop {
stop = len(subdata)
}
subdomains = append(subdomains, subdata[i:stop])
}
return strings.Join(subdomains, ".") + parent
}
func sendEnvelopeViaDNS(t *testing.T, resolver *testDNSResolver, sessionID uint32, key [32]byte, envelope *sliverpb.Envelope) {
t.Helper()
plaintext, err := proto.Marshal(envelope)
if err != nil {
t.Fatalf("marshal envelope failed: %s", err)
}
cipherCtx := implantCrypto.NewCipherContext(key)
ciphertext, err := cipherCtx.Encrypt(plaintext)
if err != nil {
t.Fatalf("encrypt envelope failed: %s", err)
}
client := dnsclient.NewDNSClient(example1, opts)
msgID := uint32(0x01000000 | sessionID)
msg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_DATA_FROM_IMPLANT,
ID: msgID,
Size: uint32(len(ciphertext)),
}
domains, err := client.SplitBuffer(msg, implantEncoders.Base32Encoder{}, ciphertext)
if err != nil {
t.Fatalf("SplitBuffer failed: %s", err)
}
subMsgs := make([]*dnspb.DNSMessage, 0, len(domains))
for _, domain := range domains {
subdata := strings.TrimSuffix(domain, example1)
subdata = strings.ReplaceAll(subdata, ".", "")
raw, err := implantEncoders.Base32Encoder{}.Decode([]byte(subdata))
if err != nil {
t.Fatalf("decode subdata failed: %s", err)
}
subMsg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(raw, subMsg); err != nil {
t.Fatalf("unmarshal submsg failed: %s", err)
}
subMsgs = append(subMsgs, subMsg)
}
shuffleDNSMsgs(subMsgs)
for _, subMsg := range subMsgs {
raw, err := proto.Marshal(subMsg)
if err != nil {
t.Fatalf("marshal submsg failed: %s", err)
}
domain, err := encodeMessageDomain(example1, subMsg)
if err != nil {
t.Fatalf("encodeMessageDomain failed: %s", err)
}
data, _, err := resolver.A(domain)
if err != nil {
t.Fatalf("resolver.A failed: %s", err)
}
if len(data) != 4 {
t.Fatalf("expected 4-byte checksum, got %d", len(data))
}
got := binary.LittleEndian.Uint32(data)
want := crc32.ChecksumIEEE(raw)
if got != want {
t.Fatalf("checksum mismatch: got %d want %d", got, want)
}
}
}
func readEnvelopeViaDNS(t *testing.T, resolver *testDNSResolver, sessionID uint32, key [32]byte, noTXT bool) (*sliverpb.Envelope, error) {
t.Helper()
pollMsg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_POLL,
ID: sessionID,
Data: randomBytes(8),
}
pollDomain, err := encodeMessageDomain(example1, pollMsg)
if err != nil {
return nil, err
}
var respData []byte
if noTXT {
respData, _, err = resolver.AAAA(pollDomain)
} else {
respData, _, err = resolver.TXT(pollDomain)
}
if err != nil {
return nil, err
}
if len(respData) == 0 {
return nil, nil
}
manifest := &dnspb.DNSMessage{}
if err := proto.Unmarshal(respData, manifest); err != nil {
return nil, err
}
if manifest.Type != dnspb.DNSMessageType_MANIFEST || manifest.Size == 0 {
return nil, nil
}
bytesPerChunk := uint32(182)
if noTXT {
bytesPerChunk = 192
}
recvBuf := make([]byte, manifest.Size)
for start := uint32(0); start < manifest.Size; start += bytesPerChunk {
stop := start + bytesPerChunk
if manifest.Size < stop {
stop = manifest.Size
}
readMsg := &dnspb.DNSMessage{
ID: manifest.ID,
Type: dnspb.DNSMessageType_DATA_TO_IMPLANT,
Start: start,
Stop: stop,
}
domain, err := encodeMessageDomain(example1, readMsg)
if err != nil {
return nil, err
}
var readData []byte
if noTXT {
readData, _, err = resolver.AAAA(domain)
} else {
readData, _, err = resolver.TXT(domain)
}
if err != nil {
return nil, err
}
recvMsg := &dnspb.DNSMessage{}
if err := proto.Unmarshal(readData, recvMsg); err != nil {
return nil, err
}
copy(recvBuf[recvMsg.Start:], recvMsg.Data)
}
plain, err := implantCrypto.NewCipherContext(key).Decrypt(recvBuf)
if err != nil && err != implantCrypto.ErrReplayAttack {
return nil, err
}
envelope := &sliverpb.Envelope{}
if err := proto.Unmarshal(plain, envelope); err != nil {
return nil, err
}
clearMsg := &dnspb.DNSMessage{
Type: dnspb.DNSMessageType_CLEAR,
ID: manifest.ID,
Data: randomBytes(8),
}
clearDomain, err := encodeMessageDomain(example1, clearMsg)
if err != nil {
return nil, err
}
if _, _, err := resolver.A(clearDomain); err != nil {
return nil, err
}
return envelope, nil
}
func randomBytes(size int) []byte {
buf := make([]byte, size)
rand.Read(buf)
return buf
}
+44 -12
View File
@@ -21,6 +21,8 @@ package generate
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/bishopfox/sliver/protobuf/clientpb"
@@ -33,6 +35,23 @@ var (
nonce = 0
)
func cleanupGeneratedArtifacts(t *testing.T, outputPath string) {
t.Helper()
if outputPath == "" {
return
}
projectDir := filepath.Dir(filepath.Dir(outputPath))
sliversDir := GetSliversDir()
rel, err := filepath.Rel(sliversDir, projectDir)
if err != nil || rel == "." || strings.HasPrefix(rel, "..") {
t.Logf("cleanup skipped for unexpected path %q", projectDir)
return
}
if err := os.RemoveAll(projectDir); err != nil {
t.Logf("cleanup failed for %s: %v", projectDir, err)
}
}
func TestSliverExecutableWindows(t *testing.T) {
// Wireguard C2
@@ -228,10 +247,11 @@ func trafficEncodersExecutable(t *testing.T, goos string, goarch string) {
debugHttpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, debugConfig)
nonce++
_, err := SliverExecutable(name, build, debugConfig, debugHttpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, debugConfig, debugHttpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
name = fmt.Sprintf("trafficEncodersProd_test%d", nonce)
prodConfig := &clientpb.ImplantConfig{
GOOS: goos,
@@ -247,10 +267,11 @@ func trafficEncodersExecutable(t *testing.T, goos string, goarch string) {
}
build, _ = GenerateConfig(name, prodConfig)
nonce++
_, err = SliverExecutable(name, build, prodConfig, debugHttpC2Config.ImplantConfig)
binPath, err = SliverExecutable(name, build, prodConfig, debugHttpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func mtlsExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -270,10 +291,11 @@ func mtlsExe(t *testing.T, goos string, goarch string, beacon bool, debug bool)
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func dnsExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -293,10 +315,11 @@ func dnsExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func httpExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -320,10 +343,11 @@ func httpExe(t *testing.T, goos string, goarch string, beacon bool, debug bool)
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func multiExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -351,10 +375,11 @@ func multiExe(t *testing.T, goos string, goarch string, beacon bool, debug bool)
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func multiWindowsService(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -381,10 +406,11 @@ func multiWindowsService(t *testing.T, goos string, goarch string, beacon bool,
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
// Pivots do not support beacon mode
@@ -408,10 +434,11 @@ func tcpPivotExe(t *testing.T, goos string, goarch string, debug bool) {
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func namedPipeExe(t *testing.T, goos string, goarch string, debug bool) {
@@ -434,10 +461,11 @@ func namedPipeExe(t *testing.T, goos string, goarch string, debug bool) {
httpC2Config := configs.GenerateDefaultHTTPC2Config()
build, _ := GenerateConfig(name, config)
nonce++
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func wireguardExe(t *testing.T, goos string, goarch string, beacon bool, debug bool) {
@@ -465,10 +493,11 @@ func wireguardExe(t *testing.T, goos string, goarch string, beacon bool, debug b
nonce++
certs.SetupWGKeys()
build, _ := GenerateConfig(name, config)
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func multiLibrary(t *testing.T, goos string, goarch string, debug bool) {
@@ -502,10 +531,11 @@ func multiLibrary(t *testing.T, goos string, goarch string, debug bool) {
httpC2Config := configs.GenerateDefaultHTTPC2Config()
nonce++
build, _ := GenerateConfig(name, config)
_, err := SliverSharedLibrary(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverSharedLibrary(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
func multiWindowsLibraryShellcode(t *testing.T, debug bool) {
@@ -537,6 +567,7 @@ func multiWindowsLibraryShellcode(t *testing.T, debug bool) {
if err != nil {
t.Fatalf("%v", err)
}
defer cleanupGeneratedArtifacts(t, binPath)
// encode bin with sgn
bin, err := os.ReadFile(binPath)
@@ -577,8 +608,9 @@ func symbolObfuscation(t *testing.T, goos string, goarch string) {
httpC2Config := configs.GenerateDefaultHTTPC2Config()
nonce++
build, _ := GenerateConfig(name, config)
_, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
binPath, err := SliverExecutable(name, build, config, httpC2Config.ImplantConfig)
if err != nil {
t.Fatalf("%v", err)
}
cleanupGeneratedArtifacts(t, binPath)
}
+215
View File
@@ -0,0 +1,215 @@
package assets
import (
"archive/tar"
"archive/zip"
"compress/gzip"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
)
func extractTarGz(archivePath, destDir string) error {
file, err := os.Open(archivePath)
if err != nil {
return fmt.Errorf("open tar.gz: %w", err)
}
defer file.Close()
gz, err := gzip.NewReader(file)
if err != nil {
return fmt.Errorf("open gzip stream: %w", err)
}
defer gz.Close()
tarReader := tar.NewReader(gz)
for {
hdr, err := tarReader.Next()
if err == io.EOF {
break
}
if err != nil {
return fmt.Errorf("read tar entry: %w", err)
}
target, err := safeJoin(destDir, hdr.Name)
if err != nil {
return err
}
mode := hdr.FileInfo().Mode()
switch hdr.Typeflag {
case tar.TypeDir:
if err := os.MkdirAll(target, mode.Perm()); err != nil {
return fmt.Errorf("create dir: %w", err)
}
case tar.TypeReg, tar.TypeRegA:
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return fmt.Errorf("create parent dir: %w", err)
}
out, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, mode.Perm())
if err != nil {
return fmt.Errorf("create file: %w", err)
}
if _, err := io.Copy(out, tarReader); err != nil {
out.Close()
return fmt.Errorf("write file: %w", err)
}
if err := out.Close(); err != nil {
return fmt.Errorf("close file: %w", err)
}
case tar.TypeSymlink:
if filepath.IsAbs(hdr.Linkname) {
return fmt.Errorf("refusing absolute symlink: %s", hdr.Linkname)
}
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return fmt.Errorf("create parent dir: %w", err)
}
if err := os.Symlink(hdr.Linkname, target); err != nil {
return fmt.Errorf("create symlink: %w", err)
}
default:
// Skip non-file entries like extended headers.
continue
}
}
return nil
}
func extractZip(archivePath, destDir string) error {
reader, err := zip.OpenReader(archivePath)
if err != nil {
return fmt.Errorf("open zip: %w", err)
}
defer reader.Close()
for _, file := range reader.File {
target, err := safeJoin(destDir, file.Name)
if err != nil {
return err
}
info := file.FileInfo()
if info.IsDir() {
if err := os.MkdirAll(target, info.Mode().Perm()); err != nil {
return fmt.Errorf("create dir: %w", err)
}
continue
}
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return fmt.Errorf("create parent dir: %w", err)
}
in, err := file.Open()
if err != nil {
return fmt.Errorf("open zip entry: %w", err)
}
out, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, info.Mode().Perm())
if err != nil {
in.Close()
return fmt.Errorf("create file: %w", err)
}
if _, err := io.Copy(out, in); err != nil {
in.Close()
out.Close()
return fmt.Errorf("write file: %w", err)
}
if err := in.Close(); err != nil {
out.Close()
return fmt.Errorf("close zip entry: %w", err)
}
if err := out.Close(); err != nil {
return fmt.Errorf("close file: %w", err)
}
}
return nil
}
func zipDir(baseDir, relRoot, destZip string) error {
root := filepath.Join(baseDir, relRoot)
if err := ensureDir(filepath.Dir(destZip)); err != nil {
return err
}
zipFile, err := os.Create(destZip)
if err != nil {
return fmt.Errorf("create zip: %w", err)
}
defer zipFile.Close()
zw := zip.NewWriter(zipFile)
defer zw.Close()
return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
info, err := d.Info()
if err != nil {
return err
}
rel, err := filepath.Rel(baseDir, path)
if err != nil {
return err
}
header, err := zip.FileInfoHeader(info)
if err != nil {
return err
}
header.Name = filepath.ToSlash(rel)
if info.IsDir() {
if !strings.HasSuffix(header.Name, "/") {
header.Name += "/"
}
header.Method = zip.Store
_, err := zw.CreateHeader(header)
return err
}
header.Method = zip.Deflate
writer, err := zw.CreateHeader(header)
if err != nil {
return err
}
file, err := os.Open(path)
if err != nil {
return err
}
if _, err := io.Copy(writer, file); err != nil {
file.Close()
return err
}
if err := file.Close(); err != nil {
return err
}
return nil
})
}
func safeJoin(root, name string) (string, error) {
cleaned := filepath.Clean(name)
if filepath.IsAbs(cleaned) {
return "", fmt.Errorf("absolute path in archive: %s", name)
}
target := filepath.Join(root, cleaned)
rel, err := filepath.Rel(root, target)
if err != nil {
return "", fmt.Errorf("resolve archive path: %w", err)
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
return "", fmt.Errorf("archive path escapes destination: %s", name)
}
return target, nil
}
+148
View File
@@ -0,0 +1,148 @@
package assets
import (
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"time"
)
type runner struct {
logger *logger
httpClient *http.Client
outputDir string
workDir string
goIndex int
zigIndex int
garbleIndex int
zigMirrors []string
}
type config struct {
verbose bool
quiet bool
noColor bool
}
// Run executes the asset generation flow.
func Run(args []string) error {
cfg, showHelp, err := parseArgs(args)
if err != nil {
return err
}
if showHelp {
fmt.Println(usage())
return nil
}
repoRoot, err := findRepoRoot()
if err != nil {
return err
}
outputDir := filepath.Join(repoRoot, "server", "assets", "fs")
if err := os.MkdirAll(outputDir, 0o755); err != nil {
return fmt.Errorf("create output dir: %w", err)
}
workDir, err := os.MkdirTemp("", "sliver-assets-")
if err != nil {
return fmt.Errorf("create work dir: %w", err)
}
log := newLogger(cfg.verbose, cfg.quiet, cfg.noColor)
r := &runner{
logger: log,
httpClient: &http.Client{Timeout: 15 * time.Minute},
outputDir: outputDir,
workDir: workDir,
}
log.Header("Sliver Assets")
log.Meta("Workdir", workDir)
log.Meta("Output", outputDir)
defer func() {
log.ClearSection()
log.Logf("clean up: %s", workDir)
os.RemoveAll(workDir)
}()
if err := r.buildGoAssets(); err != nil {
return err
}
if err := r.buildZigAssets(); err != nil {
return err
}
if err := r.buildGarbleAssets(); err != nil {
return err
}
log.ClearSection()
log.Logf("")
log.Successf("Done")
return nil
}
func parseArgs(args []string) (config, bool, error) {
cfg := config{}
showHelp := false
for _, arg := range args {
switch arg {
case "-v", "--verbose":
cfg.verbose = true
case "--no-colors":
cfg.noColor = true
case "--quiet":
cfg.quiet = true
case "-h", "--help":
showHelp = true
default:
return config{}, false, fmt.Errorf("unknown argument: %s", arg)
}
}
if cfg.quiet {
cfg.verbose = false
}
return cfg, showHelp, nil
}
func usage() string {
return "Usage: assets [--verbose] [--quiet] [--no-colors]"
}
func findRepoRoot() (string, error) {
cwd, err := os.Getwd()
if err != nil {
return "", fmt.Errorf("get working directory: %w", err)
}
dir := cwd
for {
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil {
return dir, nil
}
parent := filepath.Dir(dir)
if parent == dir {
return "", errors.New("unable to locate go.mod from current directory")
}
dir = parent
}
}
func ensureDir(path string) error {
if path == "" {
return errors.New("empty directory path")
}
return os.MkdirAll(path, 0o755)
}
func trimLeadingDot(path string) string {
return strings.TrimPrefix(path, "./")
}
+40
View File
@@ -0,0 +1,40 @@
package assets
const (
goVersion = "1.25.5"
zigVersion = "0.15.1"
garbleVersion = "1.25.5"
zigSourceParam = "source=sliver"
zigMinisignPublicKey = "RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U"
goTotal = 5
zigTotal = 5
garbleTotal = 5
)
var defaultZigMirrors = []string{
"https://pkg.machengine.org/zig",
"https://zigmirror.hryx.net/zig",
"https://zig.linus.dev/zig",
"https://zig.squirl.dev",
"https://zig.florent.dev",
"https://zig.mirror.mschae23.de/zig",
"https://zigmirror.meox.dev",
}
var goBloatPaths = []string{
"AUTHORS",
"CONTRIBUTORS",
"PATENTS",
"VERSION",
"favicon.ico",
"robots.txt",
"SECURITY.md",
"CONTRIBUTING.md",
"README.md",
"doc",
"test",
"api",
"misc",
}
+140
View File
@@ -0,0 +1,140 @@
package assets
import (
"fmt"
"io"
"net/http"
"net/url"
"os"
"path"
"path/filepath"
)
func (r *runner) downloadFile(url, dest string) error {
if err := ensureDir(filepath.Dir(dest)); err != nil {
return err
}
return r.downloadFileTo(url, dest)
}
func (r *runner) downloadFileTo(url, dest string) error {
tmpFile, err := os.CreateTemp(filepath.Dir(dest), ".download-*")
if err != nil {
return fmt.Errorf("create temp file: %w", err)
}
tmpName := tmpFile.Name()
defer func() {
tmpFile.Close()
os.Remove(tmpName)
}()
if err := r.logger.RunSpinner(fmt.Sprintf("fetch %s", downloadLabel(url)), func() error {
return r.fetchToWriter(url, tmpFile)
}); err != nil {
return err
}
if err := tmpFile.Close(); err != nil {
return fmt.Errorf("flush download: %w", err)
}
if err := moveFile(tmpName, dest); err != nil {
return fmt.Errorf("move download into place: %w", err)
}
return nil
}
func (r *runner) downloadToTemp(url, dir string) (string, error) {
if err := ensureDir(dir); err != nil {
return "", err
}
file, err := os.CreateTemp(dir, "download-*")
if err != nil {
return "", fmt.Errorf("create temp file: %w", err)
}
name := file.Name()
if err := r.logger.RunSpinner(fmt.Sprintf("fetch %s", downloadLabel(url)), func() error {
return r.fetchToWriter(url, file)
}); err != nil {
file.Close()
os.Remove(name)
return "", err
}
if err := file.Close(); err != nil {
os.Remove(name)
return "", fmt.Errorf("flush download: %w", err)
}
return name, nil
}
func (r *runner) fetchToWriter(url string, w io.Writer) error {
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
return fmt.Errorf("build request: %w", err)
}
req.Header.Set("User-Agent", "sliver-assets")
resp, err := r.httpClient.Do(req)
if err != nil {
return fmt.Errorf("download %s: %w", url, err)
}
defer resp.Body.Close()
if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
return fmt.Errorf("download %s: unexpected status %s", url, resp.Status)
}
if _, err := io.Copy(w, resp.Body); err != nil {
return fmt.Errorf("stream %s: %w", url, err)
}
return nil
}
func moveFile(src, dest string) error {
if err := os.Rename(src, dest); err == nil {
return nil
}
if err := copyFile(src, dest); err != nil {
return err
}
return os.Remove(src)
}
func copyFile(src, dest string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("open source: %w", err)
}
defer in.Close()
if err := ensureDir(filepath.Dir(dest)); err != nil {
return err
}
out, err := os.Create(dest)
if err != nil {
return fmt.Errorf("create dest: %w", err)
}
defer out.Close()
if _, err := io.Copy(out, in); err != nil {
return fmt.Errorf("copy file: %w", err)
}
if err := out.Close(); err != nil {
return fmt.Errorf("flush dest: %w", err)
}
return nil
}
func downloadLabel(rawURL string) string {
parsed, err := url.Parse(rawURL)
if err != nil {
return rawURL
}
base := path.Base(parsed.Path)
if base == "." || base == "/" || base == "" {
return rawURL
}
return base
}
+65
View File
@@ -0,0 +1,65 @@
package assets
import (
"fmt"
"path/filepath"
)
type garblePlatform struct {
os string
arch string
filename string
url string
}
func (r *runner) buildGarbleAssets() error {
r.logger.Section("Garble")
platforms := []garblePlatform{
{
os: "linux",
arch: "amd64",
filename: "garble",
url: fmt.Sprintf("https://github.com/moloch--/garble/releases/download/v%s/garble_linux-amd64", garbleVersion),
},
{
os: "linux",
arch: "arm64",
filename: "garble",
url: fmt.Sprintf("https://github.com/moloch--/garble/releases/download/v%s/garble_linux-arm64", garbleVersion),
},
{
os: "windows",
arch: "amd64",
filename: "garble.exe",
url: fmt.Sprintf("https://github.com/moloch--/garble/releases/download/v%s/garble_windows-amd64.exe", garbleVersion),
},
{
os: "darwin",
arch: "amd64",
filename: "garble",
url: fmt.Sprintf("https://github.com/moloch--/garble/releases/download/v%s/garble_darwin-amd64", garbleVersion),
},
{
os: "darwin",
arch: "arm64",
filename: "garble",
url: fmt.Sprintf("https://github.com/moloch--/garble/releases/download/v%s/garble_darwin-arm64", garbleVersion),
},
}
for _, platform := range platforms {
r.garbleIndex++
r.logger.Logf("Fetch garble %s/%s (%d/%d)", platform.os, platform.arch, r.garbleIndex, garbleTotal)
outputDir := filepath.Join(r.outputDir, platform.os, platform.arch)
if err := ensureDir(outputDir); err != nil {
return err
}
destPath := filepath.Join(outputDir, platform.filename)
if err := r.downloadFile(platform.url, destPath); err != nil {
return err
}
}
return nil
}
+157
View File
@@ -0,0 +1,157 @@
package assets
import (
"fmt"
"os"
"path/filepath"
)
type goPlatform struct {
os string
arch string
archiveExt string
includeSrc bool
toolRemoves []string
}
func (r *runner) buildGoAssets() error {
r.logger.Section("Go")
platforms := []goPlatform{
{
os: "darwin",
arch: "amd64",
archiveExt: "tar.gz",
includeSrc: true,
toolRemoves: []string{
"pkg/tool/darwin_amd64/doc",
"pkg/tool/darwin_amd64/tour",
"pkg/tool/darwin_amd64/test2json",
},
},
{
os: "darwin",
arch: "arm64",
archiveExt: "tar.gz",
includeSrc: true,
toolRemoves: []string{
"pkg/tool/darwin_arm64/doc",
"pkg/tool/darwin_arm64/tour",
"pkg/tool/darwin_arm64/test2json",
},
},
{
os: "linux",
arch: "amd64",
archiveExt: "tar.gz",
toolRemoves: []string{
"pkg/tool/linux_amd64/doc",
"pkg/tool/linux_amd64/tour",
"pkg/tool/linux_amd64/test2json",
},
},
{
os: "linux",
arch: "arm64",
archiveExt: "tar.gz",
toolRemoves: []string{
"pkg/tool/linux_arm64/doc",
"pkg/tool/linux_arm64/tour",
"pkg/tool/linux_arm64/test2json",
},
},
{
os: "windows",
arch: "amd64",
archiveExt: "zip",
toolRemoves: []string{
"pkg/tool/windows_amd64/doc.exe",
"pkg/tool/windows_amd64/tour.exe",
"pkg/tool/windows_amd64/test2json.exe",
},
},
}
for _, platform := range platforms {
label := fmt.Sprintf("%s/%s", platform.os, platform.arch)
r.goIndex++
r.logger.Logf("Fetch go %s (%d/%d)", label, r.goIndex, goTotal)
archiveName := fmt.Sprintf("go%s.%s-%s.%s", goVersion, platform.os, platform.arch, platform.archiveExt)
archiveURL := fmt.Sprintf("https://dl.google.com/go/%s", archiveName)
archivePath := filepath.Join(r.workDir, archiveName)
if err := r.downloadFile(archiveURL, archivePath); err != nil {
return err
}
r.logger.Logf("Extract go %s", label)
goDir := filepath.Join(r.workDir, "go")
if err := os.RemoveAll(goDir); err != nil {
return fmt.Errorf("remove previous go dir: %w", err)
}
if platform.archiveExt == "zip" {
if err := extractZip(archivePath, r.workDir); err != nil {
return err
}
} else {
if err := extractTarGz(archivePath, r.workDir); err != nil {
return err
}
}
if err := removePaths(goDir, goBloatPaths); err != nil {
return err
}
if platform.includeSrc {
r.logger.Logf("Pack src.zip (%s)", label)
srcZipPath := filepath.Join(r.outputDir, "src.zip")
if err := zipDir(goDir, "src", srcZipPath); err != nil {
return err
}
}
if err := os.RemoveAll(filepath.Join(goDir, "src")); err != nil {
return fmt.Errorf("remove src dir: %w", err)
}
if err := removePaths(goDir, platform.toolRemoves); err != nil {
return err
}
r.logger.Logf("Pack go.zip (%s)", label)
outputDir := filepath.Join(r.outputDir, platform.os, platform.arch)
if err := ensureDir(outputDir); err != nil {
return err
}
destZip := filepath.Join(outputDir, "go.zip")
if err := zipDir(r.workDir, "go", destZip); err != nil {
return err
}
if err := os.RemoveAll(goDir); err != nil {
return fmt.Errorf("remove go dir: %w", err)
}
if err := os.Remove(archivePath); err != nil {
return fmt.Errorf("remove go archive: %w", err)
}
}
return nil
}
func removePaths(root string, paths []string) error {
for _, path := range paths {
path = trimLeadingDot(path)
if path == "" {
continue
}
full := filepath.Join(root, path)
if err := os.RemoveAll(full); err != nil {
return fmt.Errorf("remove %s: %w", full, err)
}
}
return nil
}
+190
View File
@@ -0,0 +1,190 @@
package assets
import (
"context"
"fmt"
"io"
"os"
"strings"
huhspinner "github.com/charmbracelet/huh/spinner"
"github.com/charmbracelet/lipgloss"
"golang.org/x/term"
)
type styles struct {
title lipgloss.Style
section lipgloss.Style
tag lipgloss.Style
key lipgloss.Style
success lipgloss.Style
warn lipgloss.Style
error lipgloss.Style
muted lipgloss.Style
spinner lipgloss.Style
}
type logger struct {
verbose bool
quiet bool
out io.Writer
err io.Writer
styles styles
spinnerEnabled bool
section string
sectionCount int
}
func newLogger(verbose, quiet, noColor bool) *logger {
out := os.Stdout
err := os.Stderr
var styled styles
if noColor {
styled = styles{
title: lipgloss.NewStyle(),
section: lipgloss.NewStyle(),
tag: lipgloss.NewStyle(),
key: lipgloss.NewStyle(),
success: lipgloss.NewStyle(),
warn: lipgloss.NewStyle(),
error: lipgloss.NewStyle(),
muted: lipgloss.NewStyle(),
spinner: lipgloss.NewStyle(),
}
} else {
styled = styles{
title: lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("81")),
section: lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("75")),
tag: lipgloss.NewStyle().Foreground(lipgloss.Color("69")),
key: lipgloss.NewStyle().Foreground(lipgloss.Color("242")),
success: lipgloss.NewStyle().Foreground(lipgloss.Color("82")),
warn: lipgloss.NewStyle().Foreground(lipgloss.Color("214")),
error: lipgloss.NewStyle().Foreground(lipgloss.Color("203")),
muted: lipgloss.NewStyle().Foreground(lipgloss.Color("242")),
spinner: lipgloss.NewStyle().Foreground(lipgloss.Color("69")),
}
}
return &logger{
verbose: verbose,
quiet: quiet,
out: out,
err: err,
styles: styled,
spinnerEnabled: !verbose && !quiet && isTerminal(out),
}
}
func (l *logger) Header(title string) {
if l.quiet {
return
}
fmt.Fprintln(l.out, l.styles.title.Render(title))
}
func (l *logger) Meta(key, value string) {
if l.quiet {
return
}
label := l.styles.key.Render(fmt.Sprintf("%s:", strings.ToLower(key)))
fmt.Fprintf(l.out, "%s %s\n", label, value)
}
func (l *logger) Section(title string) {
l.section = strings.ToLower(title)
if l.quiet {
return
}
if l.sectionCount > 0 {
fmt.Fprintln(l.out)
}
l.sectionCount++
fmt.Fprintln(l.out, l.styles.section.Render(title))
}
func (l *logger) ClearSection() {
l.section = ""
}
func (l *logger) Logf(format string, args ...any) {
if l.quiet {
return
}
msg := fmt.Sprintf(format, args...)
if msg == "" {
fmt.Fprintln(l.out)
return
}
fmt.Fprintln(l.out, l.withPrefix(msg))
}
func (l *logger) VLogf(format string, args ...any) {
if l.quiet || !l.verbose {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Fprintln(l.out, l.withPrefix(l.styles.muted.Render(msg)))
}
func (l *logger) Successf(format string, args ...any) {
if l.quiet {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Fprintln(l.out, l.withPrefix(l.styles.success.Render(msg)))
}
func (l *logger) Warnf(format string, args ...any) {
if l.quiet {
return
}
msg := fmt.Sprintf(format, args...)
fmt.Fprintln(l.out, l.withPrefix(l.styles.warn.Render(msg)))
}
func (l *logger) Errorf(format string, args ...any) {
msg := fmt.Sprintf(format, args...)
fmt.Fprintln(l.err, l.withPrefix(l.styles.error.Render(msg)))
}
func (l *logger) RunSpinner(label string, action func() error) error {
if l.quiet || !l.spinnerEnabled {
return action()
}
spin := huhspinner.New().
Type(huhspinner.Pulse).
Title(label).
TitleStyle(l.styles.muted).
Style(l.styles.spinner).
Output(l.out).
ActionWithErr(func(context.Context) error {
return action()
})
return spin.Run()
}
func (l *logger) withPrefix(msg string) string {
prefix := l.prefix()
if prefix == "" {
return msg
}
return prefix + " " + msg
}
func (l *logger) prefix() string {
if l.section == "" {
return ""
}
return l.styles.tag.Render("[" + l.section + "]")
}
func isTerminal(w io.Writer) bool {
file, ok := w.(*os.File)
if !ok {
return false
}
return term.IsTerminal(int(file.Fd()))
}
+244
View File
@@ -0,0 +1,244 @@
package assets
import (
"bufio"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"time"
insecureRand "math/rand"
minisign "github.com/bishopfox/sliver/util/minisign"
)
type zigPlatform struct {
os string
arch string
remoteName string
localName string
}
func (r *runner) buildZigAssets() error {
r.logger.Section("Zig")
if err := r.loadZigMirrors(); err != nil {
return err
}
platforms := []zigPlatform{
{os: "darwin", arch: "amd64", remoteName: fmt.Sprintf("zig-x86_64-macos-%s.tar.xz", zigVersion), localName: "zig.tar.xz"},
{os: "darwin", arch: "arm64", remoteName: fmt.Sprintf("zig-aarch64-macos-%s.tar.xz", zigVersion), localName: "zig.tar.xz"},
{os: "linux", arch: "amd64", remoteName: fmt.Sprintf("zig-x86_64-linux-%s.tar.xz", zigVersion), localName: "zig.tar.xz"},
{os: "linux", arch: "arm64", remoteName: fmt.Sprintf("zig-aarch64-linux-%s.tar.xz", zigVersion), localName: "zig.tar.xz"},
{os: "windows", arch: "amd64", remoteName: fmt.Sprintf("zig-x86_64-windows-%s.zip", zigVersion), localName: "zig.zip"},
}
for _, platform := range platforms {
if err := r.downloadZig(platform); err != nil {
return err
}
}
return nil
}
func (r *runner) loadZigMirrors() error {
if len(r.zigMirrors) > 0 {
return nil
}
client := *r.httpClient
client.Timeout = 30 * time.Second
resp, err := client.Get("https://ziglang.org/download/community-mirrors.txt")
if err == nil && resp != nil {
defer resp.Body.Close()
if resp.StatusCode >= http.StatusOK && resp.StatusCode < http.StatusMultipleChoices {
mirrors, err := parseZigMirrors(resp.Body)
if err == nil && len(mirrors) > 0 {
r.zigMirrors = mirrors
return nil
}
}
}
r.zigMirrors = append([]string{}, defaultZigMirrors...)
return nil
}
func parseZigMirrors(reader io.Reader) ([]string, error) {
scanner := bufio.NewScanner(reader)
mirrors := []string{}
for scanner.Scan() {
line := scanner.Text()
if idx := strings.IndexRune(line, '#'); idx >= 0 {
line = line[:idx]
}
line = strings.TrimSpace(line)
if line == "" {
continue
}
mirrors = append(mirrors, line)
}
if err := scanner.Err(); err != nil {
return nil, err
}
return mirrors, nil
}
func (r *runner) downloadZig(platform zigPlatform) error {
outputDir := filepath.Join(r.outputDir, platform.os, platform.arch)
if err := ensureDir(outputDir); err != nil {
return err
}
destPath := filepath.Join(outputDir, platform.localName)
if err := os.RemoveAll(destPath); err != nil {
return fmt.Errorf("remove existing zig file: %w", err)
}
r.zigIndex++
mirrors := r.randomizedMirrors()
for idx, mirror := range mirrors {
mirrorBase := strings.TrimRight(mirror, "/") + "/" + zigVersion
artifactURL := appendQueryParam(mirrorBase+"/"+platform.remoteName, zigSourceParam)
signatureURL := appendQueryParam(mirrorBase+"/"+platform.remoteName+".minisig", zigSourceParam)
r.logger.Logf("Fetch zig %s/%s (%d/%d) via %s", platform.os, platform.arch, r.zigIndex, zigTotal, mirrorBase)
r.logger.VLogf(" artifact: %s", artifactURL)
r.logger.VLogf(" signature: %s", signatureURL)
artifactPath, err := r.downloadToTemp(artifactURL, r.workDir)
if err != nil {
r.logger.Errorf("Failed to download Zig artifact from %s", artifactURL)
if idx < len(mirrors)-1 {
r.logger.Warnf("Trying alternate mirror")
}
continue
}
signaturePath, err := r.downloadToTemp(signatureURL, r.workDir)
if err != nil {
os.Remove(artifactPath)
r.logger.Errorf("Failed to download Zig signature from %s", signatureURL)
if idx < len(mirrors)-1 {
r.logger.Warnf("Trying alternate mirror")
}
continue
}
if err := verifyZigSignature(artifactPath, signaturePath, platform.remoteName); err != nil {
r.logger.Errorf("Signature verification failed for %s from %s", platform.remoteName, mirrorBase)
r.logger.Errorf("Deleting corrupted download %s", artifactPath)
os.Remove(artifactPath)
os.Remove(signaturePath)
if idx < len(mirrors)-1 {
r.logger.Warnf("Trying alternate mirror")
}
continue
}
if err := moveFile(artifactPath, destPath); err != nil {
os.Remove(artifactPath)
os.Remove(signaturePath)
return err
}
os.Remove(signaturePath)
r.logger.Successf("Verified zig package -> %s", destPath)
return nil
}
return fmt.Errorf("unable to download and verify Zig package %s", platform.remoteName)
}
func (r *runner) randomizedMirrors() []string {
mirrors := append([]string{}, r.zigMirrors...)
if len(mirrors) == 0 {
return mirrors
}
insecureRand.Shuffle(len(mirrors), func(i, j int) {
mirrors[i], mirrors[j] = mirrors[j], mirrors[i]
})
return mirrors
}
func appendQueryParam(url, param string) string {
if strings.Contains(url, "?") {
return url + "&" + param
}
return url + "?" + param
}
func verifyZigSignature(artifactPath, signaturePath, expectedFile string) error {
sigData, err := os.ReadFile(signaturePath)
if err != nil {
return fmt.Errorf("read signature: %w", err)
}
pubKey, err := loadZigPublicKey()
if err != nil {
return err
}
artifact, err := os.Open(artifactPath)
if err != nil {
return fmt.Errorf("open artifact: %w", err)
}
defer artifact.Close()
reader := minisign.NewReader(artifact)
if _, err := io.Copy(io.Discard, reader); err != nil {
return fmt.Errorf("stream artifact: %w", err)
}
if !reader.Verify(pubKey, sigData) {
return errors.New("signature verification failed")
}
if err := verifyTrustedComment(sigData, expectedFile); err != nil {
return err
}
return nil
}
func loadZigPublicKey() (minisign.PublicKey, error) {
key := os.Getenv("ZIG_PUBLIC_KEY")
if key == "" {
key = zigMinisignPublicKey
}
var publicKey minisign.PublicKey
if err := publicKey.UnmarshalText([]byte(key)); err != nil {
return minisign.PublicKey{}, fmt.Errorf("invalid minisign public key: %w", err)
}
return publicKey, nil
}
func verifyTrustedComment(signature []byte, expectedFile string) error {
var sig minisign.Signature
if err := sig.UnmarshalText(signature); err != nil {
return fmt.Errorf("parse signature: %w", err)
}
fileField := trustedCommentFile(sig.TrustedComment)
if fileField == "" {
return fmt.Errorf("trusted comment missing file field: %q", sig.TrustedComment)
}
if fileField != expectedFile {
return fmt.Errorf("trusted comment file mismatch: expected %q, got %q", expectedFile, fileField)
}
return nil
}
func trustedCommentFile(trustedComment string) string {
for _, field := range strings.Fields(trustedComment) {
if after, ok := strings.CutPrefix(field, "file:"); ok {
return after
}
}
return ""
}
+15
View File
@@ -0,0 +1,15 @@
package main
import (
"fmt"
"os"
"github.com/bishopfox/sliver/util/assets"
)
func main() {
if err := assets.Run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
-68
View File
@@ -1,68 +0,0 @@
// Command zigverify validates a Zig archive against a minisign signature.
package main
import (
"errors"
"fmt"
"io"
"os"
minisign "github.com/bishopfox/sliver/util/minisign"
)
const defaultZigPublicKey = "RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U"
func loadPublicKey() (minisign.PublicKey, error) {
key := os.Getenv("ZIG_PUBLIC_KEY")
if key == "" {
key = defaultZigPublicKey
}
var publicKey minisign.PublicKey
if err := publicKey.UnmarshalText([]byte(key)); err != nil {
return minisign.PublicKey{}, fmt.Errorf("invalid minisign public key: %w", err)
}
return publicKey, nil
}
func verifySignature(artifactPath, signaturePath string) error {
artifact, err := os.Open(artifactPath)
if err != nil {
return fmt.Errorf("open artifact: %w", err)
}
defer artifact.Close()
sigData, err := os.ReadFile(signaturePath)
if err != nil {
return fmt.Errorf("read signature: %w", err)
}
pubKey, err := loadPublicKey()
if err != nil {
return err
}
reader := minisign.NewReader(artifact)
if _, err := io.Copy(io.Discard, reader); err != nil {
return fmt.Errorf("stream artifact: %w", err)
}
if !reader.Verify(pubKey, sigData) {
return errors.New("signature verification failed")
}
return nil
}
func main() {
if len(os.Args) != 3 {
fmt.Fprintf(os.Stderr, "usage: %s <artifact-path> <signature-path>\n", os.Args[0])
os.Exit(1)
}
if err := verifySignature(os.Args[1], os.Args[2]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
+22
View File
@@ -0,0 +1,22 @@
language: go
os:
- linux
- osx
- windows
go:
- go1.13.x
- go1.x
services:
- xvfb
before_install:
- export DISPLAY=:99.0
script:
- if [ "$TRAVIS_OS_NAME" = "linux" ]; then sudo apt-get install xsel; fi
- go test -v .
- if [ "$TRAVIS_OS_NAME" = "linux" ]; then sudo apt-get install xclip; fi
- go test -v .
@@ -1,4 +1,4 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Copyright (c) 2013 Ato Araki. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
@@ -10,11 +10,11 @@ notice, this list of conditions and the following disclaimer.
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
* Neither the name of @atotto. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYIGHT HOLDERS AND CONTRIBUTORS
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
@@ -25,4 +25,3 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+48
View File
@@ -0,0 +1,48 @@
[![Build Status](https://travis-ci.org/atotto/clipboard.svg?branch=master)](https://travis-ci.org/atotto/clipboard)
[![GoDoc](https://godoc.org/github.com/atotto/clipboard?status.svg)](http://godoc.org/github.com/atotto/clipboard)
# Clipboard for Go
Provide copying and pasting to the Clipboard for Go.
Build:
$ go get github.com/atotto/clipboard
Platforms:
* OSX
* Windows 7 (probably work on other Windows)
* Linux, Unix (requires 'xclip' or 'xsel' command to be installed)
Document:
* http://godoc.org/github.com/atotto/clipboard
Notes:
* Text string only
* UTF-8 text encoding only (no conversion)
TODO:
* Clipboard watcher(?)
## Commands:
paste shell command:
$ go get github.com/atotto/clipboard/cmd/gopaste
$ # example:
$ gopaste > document.txt
copy shell command:
$ go get github.com/atotto/clipboard/cmd/gocopy
$ # example:
$ cat document.txt | gocopy
+20
View File
@@ -0,0 +1,20 @@
// Copyright 2013 @atotto. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package clipboard read/write on clipboard
package clipboard
// ReadAll read string from clipboard
func ReadAll() (string, error) {
return readAll()
}
// WriteAll write string to clipboard
func WriteAll(text string) error {
return writeAll(text)
}
// Unsupported might be set true during clipboard init, to help callers decide
// whether or not to offer clipboard options.
var Unsupported bool
+52
View File
@@ -0,0 +1,52 @@
// Copyright 2013 @atotto. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin
package clipboard
import (
"os/exec"
)
var (
pasteCmdArgs = "pbpaste"
copyCmdArgs = "pbcopy"
)
func getPasteCommand() *exec.Cmd {
return exec.Command(pasteCmdArgs)
}
func getCopyCommand() *exec.Cmd {
return exec.Command(copyCmdArgs)
}
func readAll() (string, error) {
pasteCmd := getPasteCommand()
out, err := pasteCmd.Output()
if err != nil {
return "", err
}
return string(out), nil
}
func writeAll(text string) error {
copyCmd := getCopyCommand()
in, err := copyCmd.StdinPipe()
if err != nil {
return err
}
if err := copyCmd.Start(); err != nil {
return err
}
if _, err := in.Write([]byte(text)); err != nil {
return err
}
if err := in.Close(); err != nil {
return err
}
return copyCmd.Wait()
}
+42
View File
@@ -0,0 +1,42 @@
// Copyright 2013 @atotto. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build plan9
package clipboard
import (
"os"
"io/ioutil"
)
func readAll() (string, error) {
f, err := os.Open("/dev/snarf")
if err != nil {
return "", err
}
defer f.Close()
str, err := ioutil.ReadAll(f)
if err != nil {
return "", err
}
return string(str), nil
}
func writeAll(text string) error {
f, err := os.OpenFile("/dev/snarf", os.O_WRONLY, 0666)
if err != nil {
return err
}
defer f.Close()
_, err = f.Write([]byte(text))
if err != nil {
return err
}
return nil
}
+149
View File
@@ -0,0 +1,149 @@
// Copyright 2013 @atotto. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build freebsd linux netbsd openbsd solaris dragonfly
package clipboard
import (
"errors"
"os"
"os/exec"
)
const (
xsel = "xsel"
xclip = "xclip"
powershellExe = "powershell.exe"
clipExe = "clip.exe"
wlcopy = "wl-copy"
wlpaste = "wl-paste"
termuxClipboardGet = "termux-clipboard-get"
termuxClipboardSet = "termux-clipboard-set"
)
var (
Primary bool
trimDos bool
pasteCmdArgs []string
copyCmdArgs []string
xselPasteArgs = []string{xsel, "--output", "--clipboard"}
xselCopyArgs = []string{xsel, "--input", "--clipboard"}
xclipPasteArgs = []string{xclip, "-out", "-selection", "clipboard"}
xclipCopyArgs = []string{xclip, "-in", "-selection", "clipboard"}
powershellExePasteArgs = []string{powershellExe, "Get-Clipboard"}
clipExeCopyArgs = []string{clipExe}
wlpasteArgs = []string{wlpaste, "--no-newline"}
wlcopyArgs = []string{wlcopy}
termuxPasteArgs = []string{termuxClipboardGet}
termuxCopyArgs = []string{termuxClipboardSet}
missingCommands = errors.New("No clipboard utilities available. Please install xsel, xclip, wl-clipboard or Termux:API add-on for termux-clipboard-get/set.")
)
func init() {
if os.Getenv("WAYLAND_DISPLAY") != "" {
pasteCmdArgs = wlpasteArgs
copyCmdArgs = wlcopyArgs
if _, err := exec.LookPath(wlcopy); err == nil {
if _, err := exec.LookPath(wlpaste); err == nil {
return
}
}
}
pasteCmdArgs = xclipPasteArgs
copyCmdArgs = xclipCopyArgs
if _, err := exec.LookPath(xclip); err == nil {
return
}
pasteCmdArgs = xselPasteArgs
copyCmdArgs = xselCopyArgs
if _, err := exec.LookPath(xsel); err == nil {
return
}
pasteCmdArgs = termuxPasteArgs
copyCmdArgs = termuxCopyArgs
if _, err := exec.LookPath(termuxClipboardSet); err == nil {
if _, err := exec.LookPath(termuxClipboardGet); err == nil {
return
}
}
pasteCmdArgs = powershellExePasteArgs
copyCmdArgs = clipExeCopyArgs
trimDos = true
if _, err := exec.LookPath(clipExe); err == nil {
if _, err := exec.LookPath(powershellExe); err == nil {
return
}
}
Unsupported = true
}
func getPasteCommand() *exec.Cmd {
if Primary {
pasteCmdArgs = pasteCmdArgs[:1]
}
return exec.Command(pasteCmdArgs[0], pasteCmdArgs[1:]...)
}
func getCopyCommand() *exec.Cmd {
if Primary {
copyCmdArgs = copyCmdArgs[:1]
}
return exec.Command(copyCmdArgs[0], copyCmdArgs[1:]...)
}
func readAll() (string, error) {
if Unsupported {
return "", missingCommands
}
pasteCmd := getPasteCommand()
out, err := pasteCmd.Output()
if err != nil {
return "", err
}
result := string(out)
if trimDos && len(result) > 1 {
result = result[:len(result)-2]
}
return result, nil
}
func writeAll(text string) error {
if Unsupported {
return missingCommands
}
copyCmd := getCopyCommand()
in, err := copyCmd.StdinPipe()
if err != nil {
return err
}
if err := copyCmd.Start(); err != nil {
return err
}
if _, err := in.Write([]byte(text)); err != nil {
return err
}
if err := in.Close(); err != nil {
return err
}
return copyCmd.Wait()
}
+157
View File
@@ -0,0 +1,157 @@
// Copyright 2013 @atotto. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package clipboard
import (
"runtime"
"syscall"
"time"
"unsafe"
)
const (
cfUnicodetext = 13
gmemMoveable = 0x0002
)
var (
user32 = syscall.MustLoadDLL("user32")
isClipboardFormatAvailable = user32.MustFindProc("IsClipboardFormatAvailable")
openClipboard = user32.MustFindProc("OpenClipboard")
closeClipboard = user32.MustFindProc("CloseClipboard")
emptyClipboard = user32.MustFindProc("EmptyClipboard")
getClipboardData = user32.MustFindProc("GetClipboardData")
setClipboardData = user32.MustFindProc("SetClipboardData")
kernel32 = syscall.NewLazyDLL("kernel32")
globalAlloc = kernel32.NewProc("GlobalAlloc")
globalFree = kernel32.NewProc("GlobalFree")
globalLock = kernel32.NewProc("GlobalLock")
globalUnlock = kernel32.NewProc("GlobalUnlock")
lstrcpy = kernel32.NewProc("lstrcpyW")
)
// waitOpenClipboard opens the clipboard, waiting for up to a second to do so.
func waitOpenClipboard() error {
started := time.Now()
limit := started.Add(time.Second)
var r uintptr
var err error
for time.Now().Before(limit) {
r, _, err = openClipboard.Call(0)
if r != 0 {
return nil
}
time.Sleep(time.Millisecond)
}
return err
}
func readAll() (string, error) {
// LockOSThread ensure that the whole method will keep executing on the same thread from begin to end (it actually locks the goroutine thread attribution).
// Otherwise if the goroutine switch thread during execution (which is a common practice), the OpenClipboard and CloseClipboard will happen on two different threads, and it will result in a clipboard deadlock.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
if formatAvailable, _, err := isClipboardFormatAvailable.Call(cfUnicodetext); formatAvailable == 0 {
return "", err
}
err := waitOpenClipboard()
if err != nil {
return "", err
}
h, _, err := getClipboardData.Call(cfUnicodetext)
if h == 0 {
_, _, _ = closeClipboard.Call()
return "", err
}
l, _, err := globalLock.Call(h)
if l == 0 {
_, _, _ = closeClipboard.Call()
return "", err
}
text := syscall.UTF16ToString((*[1 << 20]uint16)(unsafe.Pointer(l))[:])
r, _, err := globalUnlock.Call(h)
if r == 0 {
_, _, _ = closeClipboard.Call()
return "", err
}
closed, _, err := closeClipboard.Call()
if closed == 0 {
return "", err
}
return text, nil
}
func writeAll(text string) error {
// LockOSThread ensure that the whole method will keep executing on the same thread from begin to end (it actually locks the goroutine thread attribution).
// Otherwise if the goroutine switch thread during execution (which is a common practice), the OpenClipboard and CloseClipboard will happen on two different threads, and it will result in a clipboard deadlock.
runtime.LockOSThread()
defer runtime.UnlockOSThread()
err := waitOpenClipboard()
if err != nil {
return err
}
r, _, err := emptyClipboard.Call(0)
if r == 0 {
_, _, _ = closeClipboard.Call()
return err
}
data := syscall.StringToUTF16(text)
// "If the hMem parameter identifies a memory object, the object must have
// been allocated using the function with the GMEM_MOVEABLE flag."
h, _, err := globalAlloc.Call(gmemMoveable, uintptr(len(data)*int(unsafe.Sizeof(data[0]))))
if h == 0 {
_, _, _ = closeClipboard.Call()
return err
}
defer func() {
if h != 0 {
globalFree.Call(h)
}
}()
l, _, err := globalLock.Call(h)
if l == 0 {
_, _, _ = closeClipboard.Call()
return err
}
r, _, err = lstrcpy.Call(l, uintptr(unsafe.Pointer(&data[0])))
if r == 0 {
_, _, _ = closeClipboard.Call()
return err
}
r, _, err = globalUnlock.Call(h)
if r == 0 {
if err.(syscall.Errno) != 0 {
_, _, _ = closeClipboard.Call()
return err
}
}
r, _, err = setClipboardData.Call(cfUnicodetext, h)
if r == 0 {
_, _, _ = closeClipboard.Call()
return err
}
h = 0 // suppress deferred cleanup
closed, _, err := closeClipboard.Call()
if closed == 0 {
return err
}
return nil
}
-202
View File
@@ -1,202 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-3
View File
@@ -1,3 +0,0 @@
AWS SDK for Go
Copyright 2015 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Copyright 2014-2015 Stripe, Inc.
-18
View File
@@ -1,18 +0,0 @@
package aws
// AccountIDEndpointMode controls how a resolved AWS account ID is handled for endpoint routing.
type AccountIDEndpointMode string
const (
// AccountIDEndpointModeUnset indicates the AWS account ID will not be used for endpoint routing
AccountIDEndpointModeUnset AccountIDEndpointMode = ""
// AccountIDEndpointModePreferred indicates the AWS account ID will be used for endpoint routing if present
AccountIDEndpointModePreferred = "preferred"
// AccountIDEndpointModeRequired indicates an error will be returned if the AWS account ID is not resolved from identity
AccountIDEndpointModeRequired = "required"
// AccountIDEndpointModeDisabled indicates the AWS account ID will be ignored during endpoint routing
AccountIDEndpointModeDisabled = "disabled"
)
-92
View File
@@ -1,92 +0,0 @@
// Package arn provides a parser for interacting with Amazon Resource Names.
package arn
import (
"errors"
"strings"
)
const (
arnDelimiter = ":"
arnSections = 6
arnPrefix = "arn:"
// zero-indexed
sectionPartition = 1
sectionService = 2
sectionRegion = 3
sectionAccountID = 4
sectionResource = 5
// errors
invalidPrefix = "arn: invalid prefix"
invalidSections = "arn: not enough sections"
)
// ARN captures the individual fields of an Amazon Resource Name.
// See http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html for more information.
type ARN struct {
// The partition that the resource is in. For standard AWS regions, the partition is "aws". If you have resources in
// other partitions, the partition is "aws-partitionname". For example, the partition for resources in the China
// (Beijing) region is "aws-cn".
Partition string
// The service namespace that identifies the AWS product (for example, Amazon S3, IAM, or Amazon RDS). For a list of
// namespaces, see
// http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html#genref-aws-service-namespaces.
Service string
// The region the resource resides in. Note that the ARNs for some resources do not require a region, so this
// component might be omitted.
Region string
// The ID of the AWS account that owns the resource, without the hyphens. For example, 123456789012. Note that the
// ARNs for some resources don't require an account number, so this component might be omitted.
AccountID string
// The content of this part of the ARN varies by service. It often includes an indicator of the type of resource —
// for example, an IAM user or Amazon RDS database - followed by a slash (/) or a colon (:), followed by the
// resource name itself. Some services allows paths for resource names, as described in
// http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html#arns-paths.
Resource string
}
// Parse parses an ARN into its constituent parts.
//
// Some example ARNs:
// arn:aws:elasticbeanstalk:us-east-1:123456789012:environment/My App/MyEnvironment
// arn:aws:iam::123456789012:user/David
// arn:aws:rds:eu-west-1:123456789012:db:mysql-db
// arn:aws:s3:::my_corporate_bucket/exampleobject.png
func Parse(arn string) (ARN, error) {
if !strings.HasPrefix(arn, arnPrefix) {
return ARN{}, errors.New(invalidPrefix)
}
sections := strings.SplitN(arn, arnDelimiter, arnSections)
if len(sections) != arnSections {
return ARN{}, errors.New(invalidSections)
}
return ARN{
Partition: sections[sectionPartition],
Service: sections[sectionService],
Region: sections[sectionRegion],
AccountID: sections[sectionAccountID],
Resource: sections[sectionResource],
}, nil
}
// IsARN returns whether the given string is an arn
// by looking for whether the string starts with arn:
func IsARN(arn string) bool {
return strings.HasPrefix(arn, arnPrefix) && strings.Count(arn, ":") >= arnSections-1
}
// String returns the canonical representation of the ARN
func (arn ARN) String() string {
return arnPrefix +
arn.Partition + arnDelimiter +
arn.Service + arnDelimiter +
arn.Region + arnDelimiter +
arn.AccountID + arnDelimiter +
arn.Resource
}
-33
View File
@@ -1,33 +0,0 @@
package aws
// RequestChecksumCalculation controls request checksum calculation workflow
type RequestChecksumCalculation int
const (
// RequestChecksumCalculationUnset is the unset value for RequestChecksumCalculation
RequestChecksumCalculationUnset RequestChecksumCalculation = iota
// RequestChecksumCalculationWhenSupported indicates request checksum will be calculated
// if the operation supports input checksums
RequestChecksumCalculationWhenSupported
// RequestChecksumCalculationWhenRequired indicates request checksum will be calculated
// if required by the operation or if user elects to set a checksum algorithm in request
RequestChecksumCalculationWhenRequired
)
// ResponseChecksumValidation controls response checksum validation workflow
type ResponseChecksumValidation int
const (
// ResponseChecksumValidationUnset is the unset value for ResponseChecksumValidation
ResponseChecksumValidationUnset ResponseChecksumValidation = iota
// ResponseChecksumValidationWhenSupported indicates response checksum will be validated
// if the operation supports output checksums
ResponseChecksumValidationWhenSupported
// ResponseChecksumValidationWhenRequired indicates response checksum will only
// be validated if the operation requires output checksum validation
ResponseChecksumValidationWhenRequired
)
-250
View File
@@ -1,250 +0,0 @@
package aws
import (
"net/http"
smithybearer "github.com/aws/smithy-go/auth/bearer"
"github.com/aws/smithy-go/logging"
"github.com/aws/smithy-go/middleware"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
// HTTPClient provides the interface to provide custom HTTPClients. Generally
// *http.Client is sufficient for most use cases. The HTTPClient should not
// follow 301 or 302 redirects.
type HTTPClient interface {
Do(*http.Request) (*http.Response, error)
}
// A Config provides service configuration for service clients.
type Config struct {
// The region to send requests to. This parameter is required and must
// be configured globally or on a per-client basis unless otherwise
// noted. A full list of regions is found in the "Regions and Endpoints"
// document.
//
// See http://docs.aws.amazon.com/general/latest/gr/rande.html for
// information on AWS regions.
Region string
// The credentials object to use when signing requests.
// Use the LoadDefaultConfig to load configuration from all the SDK's supported
// sources, and resolve credentials using the SDK's default credential chain.
Credentials CredentialsProvider
// The Bearer Authentication token provider to use for authenticating API
// operation calls with a Bearer Authentication token. The API clients and
// operation must support Bearer Authentication scheme in order for the
// token provider to be used. API clients created with NewFromConfig will
// automatically be configured with this option, if the API client support
// Bearer Authentication.
//
// The SDK's config.LoadDefaultConfig can automatically populate this
// option for external configuration options such as SSO session.
// https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html
BearerAuthTokenProvider smithybearer.TokenProvider
// The HTTP Client the SDK's API clients will use to invoke HTTP requests.
// The SDK defaults to a BuildableClient allowing API clients to create
// copies of the HTTP Client for service specific customizations.
//
// Use a (*http.Client) for custom behavior. Using a custom http.Client
// will prevent the SDK from modifying the HTTP client.
HTTPClient HTTPClient
// An endpoint resolver that can be used to provide or override an endpoint
// for the given service and region.
//
// See the `aws.EndpointResolver` documentation for additional usage
// information.
//
// Deprecated: See Config.EndpointResolverWithOptions
EndpointResolver EndpointResolver
// An endpoint resolver that can be used to provide or override an endpoint
// for the given service and region.
//
// When EndpointResolverWithOptions is specified, it will be used by a
// service client rather than using EndpointResolver if also specified.
//
// See the `aws.EndpointResolverWithOptions` documentation for additional
// usage information.
//
// Deprecated: with the release of endpoint resolution v2 in API clients,
// EndpointResolver and EndpointResolverWithOptions are deprecated.
// Providing a value for this field will likely prevent you from using
// newer endpoint-related service features. See API client options
// EndpointResolverV2 and BaseEndpoint.
EndpointResolverWithOptions EndpointResolverWithOptions
// RetryMaxAttempts specifies the maximum number attempts an API client
// will call an operation that fails with a retryable error.
//
// API Clients will only use this value to construct a retryer if the
// Config.Retryer member is not nil. This value will be ignored if
// Retryer is not nil.
RetryMaxAttempts int
// RetryMode specifies the retry model the API client will be created with.
//
// API Clients will only use this value to construct a retryer if the
// Config.Retryer member is not nil. This value will be ignored if
// Retryer is not nil.
RetryMode RetryMode
// Retryer is a function that provides a Retryer implementation. A Retryer
// guides how HTTP requests should be retried in case of recoverable
// failures. When nil the API client will use a default retryer.
//
// In general, the provider function should return a new instance of a
// Retryer if you are attempting to provide a consistent Retryer
// configuration across all clients. This will ensure that each client will
// be provided a new instance of the Retryer implementation, and will avoid
// issues such as sharing the same retry token bucket across services.
//
// If not nil, RetryMaxAttempts, and RetryMode will be ignored by API
// clients.
Retryer func() Retryer
// ConfigSources are the sources that were used to construct the Config.
// Allows for additional configuration to be loaded by clients.
ConfigSources []interface{}
// APIOptions provides the set of middleware mutations modify how the API
// client requests will be handled. This is useful for adding additional
// tracing data to a request, or changing behavior of the SDK's client.
APIOptions []func(*middleware.Stack) error
// The logger writer interface to write logging messages to. Defaults to
// standard error.
Logger logging.Logger
// Configures the events that will be sent to the configured logger. This
// can be used to configure the logging of signing, retries, request, and
// responses of the SDK clients.
//
// See the ClientLogMode type documentation for the complete set of logging
// modes and available configuration.
ClientLogMode ClientLogMode
// The configured DefaultsMode. If not specified, service clients will
// default to legacy.
//
// Supported modes are: auto, cross-region, in-region, legacy, mobile,
// standard
DefaultsMode DefaultsMode
// The RuntimeEnvironment configuration, only populated if the DefaultsMode
// is set to DefaultsModeAuto and is initialized by
// `config.LoadDefaultConfig`. You should not populate this structure
// programmatically, or rely on the values here within your applications.
RuntimeEnvironment RuntimeEnvironment
// AppId is an optional application specific identifier that can be set.
// When set it will be appended to the User-Agent header of every request
// in the form of App/{AppId}. This variable is sourced from environment
// variable AWS_SDK_UA_APP_ID or the shared config profile attribute sdk_ua_app_id.
// See https://docs.aws.amazon.com/sdkref/latest/guide/settings-reference.html for
// more information on environment variables and shared config settings.
AppID string
// BaseEndpoint is an intermediary transfer location to a service specific
// BaseEndpoint on a service's Options.
BaseEndpoint *string
// DisableRequestCompression toggles if an operation request could be
// compressed or not. Will be set to false by default. This variable is sourced from
// environment variable AWS_DISABLE_REQUEST_COMPRESSION or the shared config profile attribute
// disable_request_compression
DisableRequestCompression bool
// RequestMinCompressSizeBytes sets the inclusive min bytes of a request body that could be
// compressed. Will be set to 10240 by default and must be within 0 and 10485760 bytes inclusively.
// This variable is sourced from environment variable AWS_REQUEST_MIN_COMPRESSION_SIZE_BYTES or
// the shared config profile attribute request_min_compression_size_bytes
RequestMinCompressSizeBytes int64
// Controls how a resolved AWS account ID is handled for endpoint routing.
AccountIDEndpointMode AccountIDEndpointMode
// RequestChecksumCalculation determines when request checksum calculation is performed.
//
// There are two possible values for this setting:
//
// 1. RequestChecksumCalculationWhenSupported (default): The checksum is always calculated
// if the operation supports it, regardless of whether the user sets an algorithm in the request.
//
// 2. RequestChecksumCalculationWhenRequired: The checksum is only calculated if the user
// explicitly sets a checksum algorithm in the request.
//
// This setting is sourced from the environment variable AWS_REQUEST_CHECKSUM_CALCULATION
// or the shared config profile attribute "request_checksum_calculation".
RequestChecksumCalculation RequestChecksumCalculation
// ResponseChecksumValidation determines when response checksum validation is performed
//
// There are two possible values for this setting:
//
// 1. ResponseChecksumValidationWhenSupported (default): The checksum is always validated
// if the operation supports it, regardless of whether the user sets the validation mode to ENABLED in request.
//
// 2. ResponseChecksumValidationWhenRequired: The checksum is only validated if the user
// explicitly sets the validation mode to ENABLED in the request
// This variable is sourced from environment variable AWS_RESPONSE_CHECKSUM_VALIDATION or
// the shared config profile attribute "response_checksum_validation".
ResponseChecksumValidation ResponseChecksumValidation
// Registry of HTTP interceptors.
Interceptors smithyhttp.InterceptorRegistry
// Priority list of preferred auth scheme IDs.
AuthSchemePreference []string
// ServiceOptions provides service specific configuration options that will be applied
// when constructing clients for specific services. Each callback function receives the service ID
// and the service's Options struct, allowing for dynamic configuration based on the service.
ServiceOptions []func(string, any)
}
// NewConfig returns a new Config pointer that can be chained with builder
// methods to set multiple configuration values inline without using pointers.
func NewConfig() *Config {
return &Config{}
}
// Copy will return a shallow copy of the Config object.
func (c Config) Copy() Config {
cp := c
return cp
}
// EndpointDiscoveryEnableState indicates if endpoint discovery is
// enabled, disabled, auto or unset state.
//
// Default behavior (Auto or Unset) indicates operations that require endpoint
// discovery will use Endpoint Discovery by default. Operations that
// optionally use Endpoint Discovery will not use Endpoint Discovery
// unless EndpointDiscovery is explicitly enabled.
type EndpointDiscoveryEnableState uint
// Enumeration values for EndpointDiscoveryEnableState
const (
// EndpointDiscoveryUnset represents EndpointDiscoveryEnableState is unset.
// Users do not need to use this value explicitly. The behavior for unset
// is the same as for EndpointDiscoveryAuto.
EndpointDiscoveryUnset EndpointDiscoveryEnableState = iota
// EndpointDiscoveryAuto represents an AUTO state that allows endpoint
// discovery only when required by the api. This is the default
// configuration resolved by the client if endpoint discovery is neither
// enabled or disabled.
EndpointDiscoveryAuto // default state
// EndpointDiscoveryDisabled indicates client MUST not perform endpoint
// discovery even when required.
EndpointDiscoveryDisabled
// EndpointDiscoveryEnabled indicates client MUST always perform endpoint
// discovery if supported for the operation.
EndpointDiscoveryEnabled
)
-22
View File
@@ -1,22 +0,0 @@
package aws
import (
"context"
"time"
)
type suppressedContext struct {
context.Context
}
func (s *suppressedContext) Deadline() (deadline time.Time, ok bool) {
return time.Time{}, false
}
func (s *suppressedContext) Done() <-chan struct{} {
return nil
}
func (s *suppressedContext) Err() error {
return nil
}
-235
View File
@@ -1,235 +0,0 @@
package aws
import (
"context"
"fmt"
"sync/atomic"
"time"
sdkrand "github.com/aws/aws-sdk-go-v2/internal/rand"
"github.com/aws/aws-sdk-go-v2/internal/sync/singleflight"
)
// CredentialsCacheOptions are the options
type CredentialsCacheOptions struct {
// ExpiryWindow will allow the credentials to trigger refreshing prior to
// the credentials actually expiring. This is beneficial so race conditions
// with expiring credentials do not cause request to fail unexpectedly
// due to ExpiredTokenException exceptions.
//
// An ExpiryWindow of 10s would cause calls to IsExpired() to return true
// 10 seconds before the credentials are actually expired. This can cause an
// increased number of requests to refresh the credentials to occur.
//
// If ExpiryWindow is 0 or less it will be ignored.
ExpiryWindow time.Duration
// ExpiryWindowJitterFrac provides a mechanism for randomizing the
// expiration of credentials within the configured ExpiryWindow by a random
// percentage. Valid values are between 0.0 and 1.0.
//
// As an example if ExpiryWindow is 60 seconds and ExpiryWindowJitterFrac
// is 0.5 then credentials will be set to expire between 30 to 60 seconds
// prior to their actual expiration time.
//
// If ExpiryWindow is 0 or less then ExpiryWindowJitterFrac is ignored.
// If ExpiryWindowJitterFrac is 0 then no randomization will be applied to the window.
// If ExpiryWindowJitterFrac < 0 the value will be treated as 0.
// If ExpiryWindowJitterFrac > 1 the value will be treated as 1.
ExpiryWindowJitterFrac float64
}
// CredentialsCache provides caching and concurrency safe credentials retrieval
// via the provider's retrieve method.
//
// CredentialsCache will look for optional interfaces on the Provider to adjust
// how the credential cache handles credentials caching.
//
// - HandleFailRefreshCredentialsCacheStrategy - Allows provider to handle
// credential refresh failures. This could return an updated Credentials
// value, or attempt another means of retrieving credentials.
//
// - AdjustExpiresByCredentialsCacheStrategy - Allows provider to adjust how
// credentials Expires is modified. This could modify how the Credentials
// Expires is adjusted based on the CredentialsCache ExpiryWindow option.
// Such as providing a floor not to reduce the Expires below.
type CredentialsCache struct {
provider CredentialsProvider
options CredentialsCacheOptions
creds atomic.Value
sf singleflight.Group
}
// NewCredentialsCache returns a CredentialsCache that wraps provider. Provider
// is expected to not be nil. A variadic list of one or more functions can be
// provided to modify the CredentialsCache configuration. This allows for
// configuration of credential expiry window and jitter.
func NewCredentialsCache(provider CredentialsProvider, optFns ...func(options *CredentialsCacheOptions)) *CredentialsCache {
options := CredentialsCacheOptions{}
for _, fn := range optFns {
fn(&options)
}
if options.ExpiryWindow < 0 {
options.ExpiryWindow = 0
}
if options.ExpiryWindowJitterFrac < 0 {
options.ExpiryWindowJitterFrac = 0
} else if options.ExpiryWindowJitterFrac > 1 {
options.ExpiryWindowJitterFrac = 1
}
return &CredentialsCache{
provider: provider,
options: options,
}
}
// Retrieve returns the credentials. If the credentials have already been
// retrieved, and not expired the cached credentials will be returned. If the
// credentials have not been retrieved yet, or expired the provider's Retrieve
// method will be called.
//
// Returns and error if the provider's retrieve method returns an error.
func (p *CredentialsCache) Retrieve(ctx context.Context) (Credentials, error) {
if creds, ok := p.getCreds(); ok && !creds.Expired() {
return creds, nil
}
resCh := p.sf.DoChan("", func() (interface{}, error) {
return p.singleRetrieve(&suppressedContext{ctx})
})
select {
case res := <-resCh:
return res.Val.(Credentials), res.Err
case <-ctx.Done():
return Credentials{}, &RequestCanceledError{Err: ctx.Err()}
}
}
func (p *CredentialsCache) singleRetrieve(ctx context.Context) (interface{}, error) {
currCreds, ok := p.getCreds()
if ok && !currCreds.Expired() {
return currCreds, nil
}
newCreds, err := p.provider.Retrieve(ctx)
if err != nil {
handleFailToRefresh := defaultHandleFailToRefresh
if cs, ok := p.provider.(HandleFailRefreshCredentialsCacheStrategy); ok {
handleFailToRefresh = cs.HandleFailToRefresh
}
newCreds, err = handleFailToRefresh(ctx, currCreds, err)
if err != nil {
return Credentials{}, fmt.Errorf("failed to refresh cached credentials, %w", err)
}
}
if newCreds.CanExpire && p.options.ExpiryWindow > 0 {
adjustExpiresBy := defaultAdjustExpiresBy
if cs, ok := p.provider.(AdjustExpiresByCredentialsCacheStrategy); ok {
adjustExpiresBy = cs.AdjustExpiresBy
}
randFloat64, err := sdkrand.CryptoRandFloat64()
if err != nil {
return Credentials{}, fmt.Errorf("failed to get random provider, %w", err)
}
var jitter time.Duration
if p.options.ExpiryWindowJitterFrac > 0 {
jitter = time.Duration(randFloat64 *
p.options.ExpiryWindowJitterFrac * float64(p.options.ExpiryWindow))
}
newCreds, err = adjustExpiresBy(newCreds, -(p.options.ExpiryWindow - jitter))
if err != nil {
return Credentials{}, fmt.Errorf("failed to adjust credentials expires, %w", err)
}
}
p.creds.Store(&newCreds)
return newCreds, nil
}
// getCreds returns the currently stored credentials and true. Returning false
// if no credentials were stored.
func (p *CredentialsCache) getCreds() (Credentials, bool) {
v := p.creds.Load()
if v == nil {
return Credentials{}, false
}
c := v.(*Credentials)
if c == nil || !c.HasKeys() {
return Credentials{}, false
}
return *c, true
}
// ProviderSources returns a list of where the underlying credential provider
// has been sourced, if available. Returns empty if the provider doesn't implement
// the interface
func (p *CredentialsCache) ProviderSources() []CredentialSource {
asSource, ok := p.provider.(CredentialProviderSource)
if !ok {
return []CredentialSource{}
}
return asSource.ProviderSources()
}
// Invalidate will invalidate the cached credentials. The next call to Retrieve
// will cause the provider's Retrieve method to be called.
func (p *CredentialsCache) Invalidate() {
p.creds.Store((*Credentials)(nil))
}
// IsCredentialsProvider returns whether credential provider wrapped by CredentialsCache
// matches the target provider type.
func (p *CredentialsCache) IsCredentialsProvider(target CredentialsProvider) bool {
return IsCredentialsProvider(p.provider, target)
}
// HandleFailRefreshCredentialsCacheStrategy is an interface for
// CredentialsCache to allow CredentialsProvider how failed to refresh
// credentials is handled.
type HandleFailRefreshCredentialsCacheStrategy interface {
// Given the previously cached Credentials, if any, and refresh error, may
// returns new or modified set of Credentials, or error.
//
// Credential caches may use default implementation if nil.
HandleFailToRefresh(context.Context, Credentials, error) (Credentials, error)
}
// defaultHandleFailToRefresh returns the passed in error.
func defaultHandleFailToRefresh(ctx context.Context, _ Credentials, err error) (Credentials, error) {
return Credentials{}, err
}
// AdjustExpiresByCredentialsCacheStrategy is an interface for CredentialCache
// to allow CredentialsProvider to intercept adjustments to Credentials expiry
// based on expectations and use cases of CredentialsProvider.
//
// Credential caches may use default implementation if nil.
type AdjustExpiresByCredentialsCacheStrategy interface {
// Given a Credentials as input, applying any mutations and
// returning the potentially updated Credentials, or error.
AdjustExpiresBy(Credentials, time.Duration) (Credentials, error)
}
// defaultAdjustExpiresBy adds the duration to the passed in credentials Expires,
// and returns the updated credentials value. If Credentials value's CanExpire
// is false, the passed in credentials are returned unchanged.
func defaultAdjustExpiresBy(creds Credentials, dur time.Duration) (Credentials, error) {
if !creds.CanExpire {
return creds, nil
}
creds.Expires = creds.Expires.Add(dur)
return creds, nil
}
-230
View File
@@ -1,230 +0,0 @@
package aws
import (
"context"
"fmt"
"reflect"
"time"
"github.com/aws/aws-sdk-go-v2/internal/sdk"
)
// AnonymousCredentials provides a sentinel CredentialsProvider that should be
// used to instruct the SDK's signing middleware to not sign the request.
//
// Using `nil` credentials when configuring an API client will achieve the same
// result. The AnonymousCredentials type allows you to configure the SDK's
// external config loading to not attempt to source credentials from the shared
// config or environment.
//
// For example you can use this CredentialsProvider with an API client's
// Options to instruct the client not to sign a request for accessing public
// S3 bucket objects.
//
// The following example demonstrates using the AnonymousCredentials to prevent
// SDK's external config loading attempt to resolve credentials.
//
// cfg, err := config.LoadDefaultConfig(context.TODO(),
// config.WithCredentialsProvider(aws.AnonymousCredentials{}),
// )
// if err != nil {
// log.Fatalf("failed to load config, %v", err)
// }
//
// client := s3.NewFromConfig(cfg)
//
// Alternatively you can leave the API client Option's `Credential` member to
// nil. If using the `NewFromConfig` constructor you'll need to explicitly set
// the `Credentials` member to nil, if the external config resolved a
// credential provider.
//
// client := s3.New(s3.Options{
// // Credentials defaults to a nil value.
// })
//
// This can also be configured for specific operations calls too.
//
// cfg, err := config.LoadDefaultConfig(context.TODO())
// if err != nil {
// log.Fatalf("failed to load config, %v", err)
// }
//
// client := s3.NewFromConfig(config)
//
// result, err := client.GetObject(context.TODO(), s3.GetObject{
// Bucket: aws.String("example-bucket"),
// Key: aws.String("example-key"),
// }, func(o *s3.Options) {
// o.Credentials = nil
// // Or
// o.Credentials = aws.AnonymousCredentials{}
// })
type AnonymousCredentials struct{}
// Retrieve implements the CredentialsProvider interface, but will always
// return error, and cannot be used to sign a request. The AnonymousCredentials
// type is used as a sentinel type instructing the AWS request signing
// middleware to not sign a request.
func (AnonymousCredentials) Retrieve(context.Context) (Credentials, error) {
return Credentials{Source: "AnonymousCredentials"},
fmt.Errorf("the AnonymousCredentials is not a valid credential provider, and cannot be used to sign AWS requests with")
}
// CredentialSource is the source of the credential provider.
// A provider can have multiple credential sources: For example, a provider that reads a profile, calls ECS to
// get credentials and then assumes a role using STS will have all these as part of its provider chain.
type CredentialSource int
const (
// CredentialSourceUndefined is the sentinel zero value
CredentialSourceUndefined CredentialSource = iota
// CredentialSourceCode credentials resolved from code, cli parameters, session object, or client instance
CredentialSourceCode
// CredentialSourceEnvVars credentials resolved from environment variables
CredentialSourceEnvVars
// CredentialSourceEnvVarsSTSWebIDToken credentials resolved from environment variables for assuming a role with STS using a web identity token
CredentialSourceEnvVarsSTSWebIDToken
// CredentialSourceSTSAssumeRole credentials resolved from STS using AssumeRole
CredentialSourceSTSAssumeRole
// CredentialSourceSTSAssumeRoleSaml credentials resolved from STS using assume role with SAML
CredentialSourceSTSAssumeRoleSaml
// CredentialSourceSTSAssumeRoleWebID credentials resolved from STS using assume role with web identity
CredentialSourceSTSAssumeRoleWebID
// CredentialSourceSTSFederationToken credentials resolved from STS using a federation token
CredentialSourceSTSFederationToken
// CredentialSourceSTSSessionToken credentials resolved from STS using a session token S
CredentialSourceSTSSessionToken
// CredentialSourceProfile credentials resolved from a config file(s) profile with static credentials
CredentialSourceProfile
// CredentialSourceProfileSourceProfile credentials resolved from a source profile in a config file(s) profile
CredentialSourceProfileSourceProfile
// CredentialSourceProfileNamedProvider credentials resolved from a named provider in a config file(s) profile (like EcsContainer)
CredentialSourceProfileNamedProvider
// CredentialSourceProfileSTSWebIDToken credentials resolved from configuration for assuming a role with STS using web identity token in a config file(s) profile
CredentialSourceProfileSTSWebIDToken
// CredentialSourceProfileSSO credentials resolved from an SSO session in a config file(s) profile
CredentialSourceProfileSSO
// CredentialSourceSSO credentials resolved from an SSO session
CredentialSourceSSO
// CredentialSourceProfileSSOLegacy credentials resolved from an SSO session in a config file(s) profile using legacy format
CredentialSourceProfileSSOLegacy
// CredentialSourceSSOLegacy credentials resolved from an SSO session using legacy format
CredentialSourceSSOLegacy
// CredentialSourceProfileProcess credentials resolved from a process in a config file(s) profile
CredentialSourceProfileProcess
// CredentialSourceProcess credentials resolved from a process
CredentialSourceProcess
// CredentialSourceHTTP credentials resolved from an HTTP endpoint
CredentialSourceHTTP
// CredentialSourceIMDS credentials resolved from the instance metadata service (IMDS)
CredentialSourceIMDS
)
// A Credentials is the AWS credentials value for individual credential fields.
type Credentials struct {
// AWS Access key ID
AccessKeyID string
// AWS Secret Access Key
SecretAccessKey string
// AWS Session Token
SessionToken string
// Source of the credentials
Source string
// States if the credentials can expire or not.
CanExpire bool
// The time the credentials will expire at. Should be ignored if CanExpire
// is false.
Expires time.Time
// The ID of the account for the credentials.
AccountID string
}
// Expired returns if the credentials have expired.
func (v Credentials) Expired() bool {
if v.CanExpire {
// Calling Round(0) on the current time will truncate the monotonic
// reading only. Ensures credential expiry time is always based on
// reported wall-clock time.
return !v.Expires.After(sdk.NowTime().Round(0))
}
return false
}
// HasKeys returns if the credentials keys are set.
func (v Credentials) HasKeys() bool {
return len(v.AccessKeyID) > 0 && len(v.SecretAccessKey) > 0
}
// A CredentialsProvider is the interface for any component which will provide
// credentials Credentials. A CredentialsProvider is required to manage its own
// Expired state, and what to be expired means.
//
// A credentials provider implementation can be wrapped with a CredentialCache
// to cache the credential value retrieved. Without the cache the SDK will
// attempt to retrieve the credentials for every request.
type CredentialsProvider interface {
// Retrieve returns nil if it successfully retrieved the value.
// Error is returned if the value were not obtainable, or empty.
Retrieve(ctx context.Context) (Credentials, error)
}
// CredentialProviderSource allows any credential provider to track
// all providers where a credential provider were sourced. For example, if the credentials came from a
// call to a role specified in the profile, this method will give the whole breadcrumb trail
type CredentialProviderSource interface {
ProviderSources() []CredentialSource
}
// CredentialsProviderFunc provides a helper wrapping a function value to
// satisfy the CredentialsProvider interface.
type CredentialsProviderFunc func(context.Context) (Credentials, error)
// Retrieve delegates to the function value the CredentialsProviderFunc wraps.
func (fn CredentialsProviderFunc) Retrieve(ctx context.Context) (Credentials, error) {
return fn(ctx)
}
type isCredentialsProvider interface {
IsCredentialsProvider(CredentialsProvider) bool
}
// IsCredentialsProvider returns whether the target CredentialProvider is the same type as provider when comparing the
// implementation type.
//
// If provider has a method IsCredentialsProvider(CredentialsProvider) bool it will be responsible for validating
// whether target matches the credential provider type.
//
// When comparing the CredentialProvider implementations provider and target for equality, the following rules are used:
//
// If provider is of type T and target is of type V, true if type *T is the same as type *V, otherwise false
// If provider is of type *T and target is of type V, true if type *T is the same as type *V, otherwise false
// If provider is of type T and target is of type *V, true if type *T is the same as type *V, otherwise false
// If provider is of type *T and target is of type *V,true if type *T is the same as type *V, otherwise false
func IsCredentialsProvider(provider, target CredentialsProvider) bool {
if target == nil || provider == nil {
return provider == target
}
if x, ok := provider.(isCredentialsProvider); ok {
return x.IsCredentialsProvider(target)
}
targetType := reflect.TypeOf(target)
if targetType.Kind() != reflect.Ptr {
targetType = reflect.PtrTo(targetType)
}
providerType := reflect.TypeOf(provider)
if providerType.Kind() != reflect.Ptr {
providerType = reflect.PtrTo(providerType)
}
return targetType.AssignableTo(providerType)
}
-38
View File
@@ -1,38 +0,0 @@
package defaults
import (
"github.com/aws/aws-sdk-go-v2/aws"
"runtime"
"strings"
)
var getGOOS = func() string {
return runtime.GOOS
}
// ResolveDefaultsModeAuto is used to determine the effective aws.DefaultsMode when the mode
// is set to aws.DefaultsModeAuto.
func ResolveDefaultsModeAuto(region string, environment aws.RuntimeEnvironment) aws.DefaultsMode {
goos := getGOOS()
if goos == "android" || goos == "ios" {
return aws.DefaultsModeMobile
}
var currentRegion string
if len(environment.EnvironmentIdentifier) > 0 {
currentRegion = environment.Region
}
if len(currentRegion) == 0 && len(environment.EC2InstanceMetadataRegion) > 0 {
currentRegion = environment.EC2InstanceMetadataRegion
}
if len(region) > 0 && len(currentRegion) > 0 {
if strings.EqualFold(region, currentRegion) {
return aws.DefaultsModeInRegion
}
return aws.DefaultsModeCrossRegion
}
return aws.DefaultsModeStandard
}
-43
View File
@@ -1,43 +0,0 @@
package defaults
import (
"time"
"github.com/aws/aws-sdk-go-v2/aws"
)
// Configuration is the set of SDK configuration options that are determined based
// on the configured DefaultsMode.
type Configuration struct {
// RetryMode is the configuration's default retry mode API clients should
// use for constructing a Retryer.
RetryMode aws.RetryMode
// ConnectTimeout is the maximum amount of time a dial will wait for
// a connect to complete.
//
// See https://pkg.go.dev/net#Dialer.Timeout
ConnectTimeout *time.Duration
// TLSNegotiationTimeout specifies the maximum amount of time waiting to
// wait for a TLS handshake.
//
// See https://pkg.go.dev/net/http#Transport.TLSHandshakeTimeout
TLSNegotiationTimeout *time.Duration
}
// GetConnectTimeout returns the ConnectTimeout value, returns false if the value is not set.
func (c *Configuration) GetConnectTimeout() (time.Duration, bool) {
if c.ConnectTimeout == nil {
return 0, false
}
return *c.ConnectTimeout, true
}
// GetTLSNegotiationTimeout returns the TLSNegotiationTimeout value, returns false if the value is not set.
func (c *Configuration) GetTLSNegotiationTimeout() (time.Duration, bool) {
if c.TLSNegotiationTimeout == nil {
return 0, false
}
return *c.TLSNegotiationTimeout, true
}
-50
View File
@@ -1,50 +0,0 @@
// Code generated by github.com/aws/aws-sdk-go-v2/internal/codegen/cmd/defaultsconfig. DO NOT EDIT.
package defaults
import (
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"time"
)
// GetModeConfiguration returns the default Configuration descriptor for the given mode.
//
// Supports the following modes: cross-region, in-region, mobile, standard
func GetModeConfiguration(mode aws.DefaultsMode) (Configuration, error) {
var mv aws.DefaultsMode
mv.SetFromString(string(mode))
switch mv {
case aws.DefaultsModeCrossRegion:
settings := Configuration{
ConnectTimeout: aws.Duration(3100 * time.Millisecond),
RetryMode: aws.RetryMode("standard"),
TLSNegotiationTimeout: aws.Duration(3100 * time.Millisecond),
}
return settings, nil
case aws.DefaultsModeInRegion:
settings := Configuration{
ConnectTimeout: aws.Duration(1100 * time.Millisecond),
RetryMode: aws.RetryMode("standard"),
TLSNegotiationTimeout: aws.Duration(1100 * time.Millisecond),
}
return settings, nil
case aws.DefaultsModeMobile:
settings := Configuration{
ConnectTimeout: aws.Duration(30000 * time.Millisecond),
RetryMode: aws.RetryMode("standard"),
TLSNegotiationTimeout: aws.Duration(30000 * time.Millisecond),
}
return settings, nil
case aws.DefaultsModeStandard:
settings := Configuration{
ConnectTimeout: aws.Duration(3100 * time.Millisecond),
RetryMode: aws.RetryMode("standard"),
TLSNegotiationTimeout: aws.Duration(3100 * time.Millisecond),
}
return settings, nil
default:
return Configuration{}, fmt.Errorf("unsupported defaults mode: %v", mode)
}
}
-2
View File
@@ -1,2 +0,0 @@
// Package defaults provides recommended configuration values for AWS SDKs and CLIs.
package defaults
-95
View File
@@ -1,95 +0,0 @@
// Code generated by github.com/aws/aws-sdk-go-v2/internal/codegen/cmd/defaultsmode. DO NOT EDIT.
package aws
import (
"strings"
)
// DefaultsMode is the SDK defaults mode setting.
type DefaultsMode string
// The DefaultsMode constants.
const (
// DefaultsModeAuto is an experimental mode that builds on the standard mode.
// The SDK will attempt to discover the execution environment to determine the
// appropriate settings automatically.
//
// Note that the auto detection is heuristics-based and does not guarantee 100%
// accuracy. STANDARD mode will be used if the execution environment cannot
// be determined. The auto detection might query EC2 Instance Metadata service
// (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html),
// which might introduce latency. Therefore we recommend choosing an explicit
// defaults_mode instead if startup latency is critical to your application
DefaultsModeAuto DefaultsMode = "auto"
// DefaultsModeCrossRegion builds on the standard mode and includes optimization
// tailored for applications which call AWS services in a different region
//
// Note that the default values vended from this mode might change as best practices
// may evolve. As a result, it is encouraged to perform tests when upgrading
// the SDK
DefaultsModeCrossRegion DefaultsMode = "cross-region"
// DefaultsModeInRegion builds on the standard mode and includes optimization
// tailored for applications which call AWS services from within the same AWS
// region
//
// Note that the default values vended from this mode might change as best practices
// may evolve. As a result, it is encouraged to perform tests when upgrading
// the SDK
DefaultsModeInRegion DefaultsMode = "in-region"
// DefaultsModeLegacy provides default settings that vary per SDK and were used
// prior to establishment of defaults_mode
DefaultsModeLegacy DefaultsMode = "legacy"
// DefaultsModeMobile builds on the standard mode and includes optimization
// tailored for mobile applications
//
// Note that the default values vended from this mode might change as best practices
// may evolve. As a result, it is encouraged to perform tests when upgrading
// the SDK
DefaultsModeMobile DefaultsMode = "mobile"
// DefaultsModeStandard provides the latest recommended default values that
// should be safe to run in most scenarios
//
// Note that the default values vended from this mode might change as best practices
// may evolve. As a result, it is encouraged to perform tests when upgrading
// the SDK
DefaultsModeStandard DefaultsMode = "standard"
)
// SetFromString sets the DefaultsMode value to one of the pre-defined constants that matches
// the provided string when compared using EqualFold. If the value does not match a known
// constant it will be set to as-is and the function will return false. As a special case, if the
// provided value is a zero-length string, the mode will be set to LegacyDefaultsMode.
func (d *DefaultsMode) SetFromString(v string) (ok bool) {
switch {
case strings.EqualFold(v, string(DefaultsModeAuto)):
*d = DefaultsModeAuto
ok = true
case strings.EqualFold(v, string(DefaultsModeCrossRegion)):
*d = DefaultsModeCrossRegion
ok = true
case strings.EqualFold(v, string(DefaultsModeInRegion)):
*d = DefaultsModeInRegion
ok = true
case strings.EqualFold(v, string(DefaultsModeLegacy)):
*d = DefaultsModeLegacy
ok = true
case strings.EqualFold(v, string(DefaultsModeMobile)):
*d = DefaultsModeMobile
ok = true
case strings.EqualFold(v, string(DefaultsModeStandard)):
*d = DefaultsModeStandard
ok = true
case len(v) == 0:
*d = DefaultsModeLegacy
ok = true
default:
*d = DefaultsMode(v)
}
return ok
}
-62
View File
@@ -1,62 +0,0 @@
// Package aws provides the core SDK's utilities and shared types. Use this package's
// utilities to simplify setting and reading API operations parameters.
//
// # Value and Pointer Conversion Utilities
//
// This package includes a helper conversion utility for each scalar type the SDK's
// API use. These utilities make getting a pointer of the scalar, and dereferencing
// a pointer easier.
//
// Each conversion utility comes in two forms. Value to Pointer and Pointer to Value.
// The Pointer to value will safely dereference the pointer and return its value.
// If the pointer was nil, the scalar's zero value will be returned.
//
// The value to pointer functions will be named after the scalar type. So get a
// *string from a string value use the "String" function. This makes it easy to
// to get pointer of a literal string value, because getting the address of a
// literal requires assigning the value to a variable first.
//
// var strPtr *string
//
// // Without the SDK's conversion functions
// str := "my string"
// strPtr = &str
//
// // With the SDK's conversion functions
// strPtr = aws.String("my string")
//
// // Convert *string to string value
// str = aws.ToString(strPtr)
//
// In addition to scalars the aws package also includes conversion utilities for
// map and slice for commonly types used in API parameters. The map and slice
// conversion functions use similar naming pattern as the scalar conversion
// functions.
//
// var strPtrs []*string
// var strs []string = []string{"Go", "Gophers", "Go"}
//
// // Convert []string to []*string
// strPtrs = aws.StringSlice(strs)
//
// // Convert []*string to []string
// strs = aws.ToStringSlice(strPtrs)
//
// # SDK Default HTTP Client
//
// The SDK will use the http.DefaultClient if a HTTP client is not provided to
// the SDK's Session, or service client constructor. This means that if the
// http.DefaultClient is modified by other components of your application the
// modifications will be picked up by the SDK as well.
//
// In some cases this might be intended, but it is a better practice to create
// a custom HTTP Client to share explicitly through your application. You can
// configure the SDK to use the custom HTTP Client by setting the HTTPClient
// value of the SDK's Config type when creating a Session or service client.
package aws
// generate.go uses a build tag of "ignore", go run doesn't need to specify
// this because go run ignores all build flags when running a go file directly.
//go:generate go run -tags codegen generate.go
//go:generate go run -tags codegen logging_generate.go
//go:generate gofmt -w -s .
-247
View File
@@ -1,247 +0,0 @@
package aws
import (
"fmt"
)
// DualStackEndpointState is a constant to describe the dual-stack endpoint resolution behavior.
type DualStackEndpointState uint
const (
// DualStackEndpointStateUnset is the default value behavior for dual-stack endpoint resolution.
DualStackEndpointStateUnset DualStackEndpointState = iota
// DualStackEndpointStateEnabled enables dual-stack endpoint resolution for service endpoints.
DualStackEndpointStateEnabled
// DualStackEndpointStateDisabled disables dual-stack endpoint resolution for endpoints.
DualStackEndpointStateDisabled
)
// GetUseDualStackEndpoint takes a service's EndpointResolverOptions and returns the UseDualStackEndpoint value.
// Returns boolean false if the provided options does not have a method to retrieve the DualStackEndpointState.
func GetUseDualStackEndpoint(options ...interface{}) (value DualStackEndpointState, found bool) {
type iface interface {
GetUseDualStackEndpoint() DualStackEndpointState
}
for _, option := range options {
if i, ok := option.(iface); ok {
value = i.GetUseDualStackEndpoint()
found = true
break
}
}
return value, found
}
// FIPSEndpointState is a constant to describe the FIPS endpoint resolution behavior.
type FIPSEndpointState uint
const (
// FIPSEndpointStateUnset is the default value behavior for FIPS endpoint resolution.
FIPSEndpointStateUnset FIPSEndpointState = iota
// FIPSEndpointStateEnabled enables FIPS endpoint resolution for service endpoints.
FIPSEndpointStateEnabled
// FIPSEndpointStateDisabled disables FIPS endpoint resolution for endpoints.
FIPSEndpointStateDisabled
)
// GetUseFIPSEndpoint takes a service's EndpointResolverOptions and returns the UseDualStackEndpoint value.
// Returns boolean false if the provided options does not have a method to retrieve the DualStackEndpointState.
func GetUseFIPSEndpoint(options ...interface{}) (value FIPSEndpointState, found bool) {
type iface interface {
GetUseFIPSEndpoint() FIPSEndpointState
}
for _, option := range options {
if i, ok := option.(iface); ok {
value = i.GetUseFIPSEndpoint()
found = true
break
}
}
return value, found
}
// Endpoint represents the endpoint a service client should make API operation
// calls to.
//
// The SDK will automatically resolve these endpoints per API client using an
// internal endpoint resolvers. If you'd like to provide custom endpoint
// resolving behavior you can implement the EndpointResolver interface.
//
// Deprecated: This structure was used with the global [EndpointResolver]
// interface, which has been deprecated in favor of service-specific endpoint
// resolution. See the deprecation docs on that interface for more information.
type Endpoint struct {
// The base URL endpoint the SDK API clients will use to make API calls to.
// The SDK will suffix URI path and query elements to this endpoint.
URL string
// Specifies if the endpoint's hostname can be modified by the SDK's API
// client.
//
// If the hostname is mutable the SDK API clients may modify any part of
// the hostname based on the requirements of the API, (e.g. adding, or
// removing content in the hostname). Such as, Amazon S3 API client
// prefixing "bucketname" to the hostname, or changing the
// hostname service name component from "s3." to "s3-accesspoint.dualstack."
// for the dualstack endpoint of an S3 Accesspoint resource.
//
// Care should be taken when providing a custom endpoint for an API. If the
// endpoint hostname is mutable, and the client cannot modify the endpoint
// correctly, the operation call will most likely fail, or have undefined
// behavior.
//
// If hostname is immutable, the SDK API clients will not modify the
// hostname of the URL. This may cause the API client not to function
// correctly if the API requires the operation specific hostname values
// to be used by the client.
//
// This flag does not modify the API client's behavior if this endpoint
// will be used instead of Endpoint Discovery, or if the endpoint will be
// used to perform Endpoint Discovery. That behavior is configured via the
// API Client's Options.
HostnameImmutable bool
// The AWS partition the endpoint belongs to.
PartitionID string
// The service name that should be used for signing the requests to the
// endpoint.
SigningName string
// The region that should be used for signing the request to the endpoint.
SigningRegion string
// The signing method that should be used for signing the requests to the
// endpoint.
SigningMethod string
// The source of the Endpoint. By default, this will be EndpointSourceServiceMetadata.
// When providing a custom endpoint, you should set the source as EndpointSourceCustom.
// If source is not provided when providing a custom endpoint, the SDK may not
// perform required host mutations correctly. Source should be used along with
// HostnameImmutable property as per the usage requirement.
Source EndpointSource
}
// EndpointSource is the endpoint source type.
//
// Deprecated: The global [Endpoint] structure is deprecated.
type EndpointSource int
const (
// EndpointSourceServiceMetadata denotes service modeled endpoint metadata is used as Endpoint Source.
EndpointSourceServiceMetadata EndpointSource = iota
// EndpointSourceCustom denotes endpoint is a custom endpoint. This source should be used when
// user provides a custom endpoint to be used by the SDK.
EndpointSourceCustom
)
// EndpointNotFoundError is a sentinel error to indicate that the
// EndpointResolver implementation was unable to resolve an endpoint for the
// given service and region. Resolvers should use this to indicate that an API
// client should fallback and attempt to use it's internal default resolver to
// resolve the endpoint.
type EndpointNotFoundError struct {
Err error
}
// Error is the error message.
func (e *EndpointNotFoundError) Error() string {
return fmt.Sprintf("endpoint not found, %v", e.Err)
}
// Unwrap returns the underlying error.
func (e *EndpointNotFoundError) Unwrap() error {
return e.Err
}
// EndpointResolver is an endpoint resolver that can be used to provide or
// override an endpoint for the given service and region. API clients will
// attempt to use the EndpointResolver first to resolve an endpoint if
// available. If the EndpointResolver returns an EndpointNotFoundError error,
// API clients will fallback to attempting to resolve the endpoint using its
// internal default endpoint resolver.
//
// Deprecated: The global endpoint resolution interface is deprecated. The API
// for endpoint resolution is now unique to each service and is set via the
// EndpointResolverV2 field on service client options. Setting a value for
// EndpointResolver on aws.Config or service client options will prevent you
// from using any endpoint-related service features released after the
// introduction of EndpointResolverV2. You may also encounter broken or
// unexpected behavior when using the old global interface with services that
// use many endpoint-related customizations such as S3.
type EndpointResolver interface {
ResolveEndpoint(service, region string) (Endpoint, error)
}
// EndpointResolverFunc wraps a function to satisfy the EndpointResolver interface.
//
// Deprecated: The global endpoint resolution interface is deprecated. See
// deprecation docs on [EndpointResolver].
type EndpointResolverFunc func(service, region string) (Endpoint, error)
// ResolveEndpoint calls the wrapped function and returns the results.
func (e EndpointResolverFunc) ResolveEndpoint(service, region string) (Endpoint, error) {
return e(service, region)
}
// EndpointResolverWithOptions is an endpoint resolver that can be used to provide or
// override an endpoint for the given service, region, and the service client's EndpointOptions. API clients will
// attempt to use the EndpointResolverWithOptions first to resolve an endpoint if
// available. If the EndpointResolverWithOptions returns an EndpointNotFoundError error,
// API clients will fallback to attempting to resolve the endpoint using its
// internal default endpoint resolver.
//
// Deprecated: The global endpoint resolution interface is deprecated. See
// deprecation docs on [EndpointResolver].
type EndpointResolverWithOptions interface {
ResolveEndpoint(service, region string, options ...interface{}) (Endpoint, error)
}
// EndpointResolverWithOptionsFunc wraps a function to satisfy the EndpointResolverWithOptions interface.
//
// Deprecated: The global endpoint resolution interface is deprecated. See
// deprecation docs on [EndpointResolver].
type EndpointResolverWithOptionsFunc func(service, region string, options ...interface{}) (Endpoint, error)
// ResolveEndpoint calls the wrapped function and returns the results.
func (e EndpointResolverWithOptionsFunc) ResolveEndpoint(service, region string, options ...interface{}) (Endpoint, error) {
return e(service, region, options...)
}
// GetDisableHTTPS takes a service's EndpointResolverOptions and returns the DisableHTTPS value.
// Returns boolean false if the provided options does not have a method to retrieve the DisableHTTPS.
func GetDisableHTTPS(options ...interface{}) (value bool, found bool) {
type iface interface {
GetDisableHTTPS() bool
}
for _, option := range options {
if i, ok := option.(iface); ok {
value = i.GetDisableHTTPS()
found = true
break
}
}
return value, found
}
// GetResolvedRegion takes a service's EndpointResolverOptions and returns the ResolvedRegion value.
// Returns boolean false if the provided options does not have a method to retrieve the ResolvedRegion.
func GetResolvedRegion(options ...interface{}) (value string, found bool) {
type iface interface {
GetResolvedRegion() string
}
for _, option := range options {
if i, ok := option.(iface); ok {
value = i.GetResolvedRegion()
found = true
break
}
}
return value, found
}
-9
View File
@@ -1,9 +0,0 @@
package aws
// MissingRegionError is an error that is returned if region configuration
// value was not found.
type MissingRegionError struct{}
func (*MissingRegionError) Error() string {
return "an AWS region is required, but was not found"
}
-365
View File
@@ -1,365 +0,0 @@
// Code generated by aws/generate.go DO NOT EDIT.
package aws
import (
"github.com/aws/smithy-go/ptr"
"time"
)
// ToBool returns bool value dereferenced if the passed
// in pointer was not nil. Returns a bool zero value if the
// pointer was nil.
func ToBool(p *bool) (v bool) {
return ptr.ToBool(p)
}
// ToBoolSlice returns a slice of bool values, that are
// dereferenced if the passed in pointer was not nil. Returns a bool
// zero value if the pointer was nil.
func ToBoolSlice(vs []*bool) []bool {
return ptr.ToBoolSlice(vs)
}
// ToBoolMap returns a map of bool values, that are
// dereferenced if the passed in pointer was not nil. The bool
// zero value is used if the pointer was nil.
func ToBoolMap(vs map[string]*bool) map[string]bool {
return ptr.ToBoolMap(vs)
}
// ToByte returns byte value dereferenced if the passed
// in pointer was not nil. Returns a byte zero value if the
// pointer was nil.
func ToByte(p *byte) (v byte) {
return ptr.ToByte(p)
}
// ToByteSlice returns a slice of byte values, that are
// dereferenced if the passed in pointer was not nil. Returns a byte
// zero value if the pointer was nil.
func ToByteSlice(vs []*byte) []byte {
return ptr.ToByteSlice(vs)
}
// ToByteMap returns a map of byte values, that are
// dereferenced if the passed in pointer was not nil. The byte
// zero value is used if the pointer was nil.
func ToByteMap(vs map[string]*byte) map[string]byte {
return ptr.ToByteMap(vs)
}
// ToString returns string value dereferenced if the passed
// in pointer was not nil. Returns a string zero value if the
// pointer was nil.
func ToString(p *string) (v string) {
return ptr.ToString(p)
}
// ToStringSlice returns a slice of string values, that are
// dereferenced if the passed in pointer was not nil. Returns a string
// zero value if the pointer was nil.
func ToStringSlice(vs []*string) []string {
return ptr.ToStringSlice(vs)
}
// ToStringMap returns a map of string values, that are
// dereferenced if the passed in pointer was not nil. The string
// zero value is used if the pointer was nil.
func ToStringMap(vs map[string]*string) map[string]string {
return ptr.ToStringMap(vs)
}
// ToInt returns int value dereferenced if the passed
// in pointer was not nil. Returns a int zero value if the
// pointer was nil.
func ToInt(p *int) (v int) {
return ptr.ToInt(p)
}
// ToIntSlice returns a slice of int values, that are
// dereferenced if the passed in pointer was not nil. Returns a int
// zero value if the pointer was nil.
func ToIntSlice(vs []*int) []int {
return ptr.ToIntSlice(vs)
}
// ToIntMap returns a map of int values, that are
// dereferenced if the passed in pointer was not nil. The int
// zero value is used if the pointer was nil.
func ToIntMap(vs map[string]*int) map[string]int {
return ptr.ToIntMap(vs)
}
// ToInt8 returns int8 value dereferenced if the passed
// in pointer was not nil. Returns a int8 zero value if the
// pointer was nil.
func ToInt8(p *int8) (v int8) {
return ptr.ToInt8(p)
}
// ToInt8Slice returns a slice of int8 values, that are
// dereferenced if the passed in pointer was not nil. Returns a int8
// zero value if the pointer was nil.
func ToInt8Slice(vs []*int8) []int8 {
return ptr.ToInt8Slice(vs)
}
// ToInt8Map returns a map of int8 values, that are
// dereferenced if the passed in pointer was not nil. The int8
// zero value is used if the pointer was nil.
func ToInt8Map(vs map[string]*int8) map[string]int8 {
return ptr.ToInt8Map(vs)
}
// ToInt16 returns int16 value dereferenced if the passed
// in pointer was not nil. Returns a int16 zero value if the
// pointer was nil.
func ToInt16(p *int16) (v int16) {
return ptr.ToInt16(p)
}
// ToInt16Slice returns a slice of int16 values, that are
// dereferenced if the passed in pointer was not nil. Returns a int16
// zero value if the pointer was nil.
func ToInt16Slice(vs []*int16) []int16 {
return ptr.ToInt16Slice(vs)
}
// ToInt16Map returns a map of int16 values, that are
// dereferenced if the passed in pointer was not nil. The int16
// zero value is used if the pointer was nil.
func ToInt16Map(vs map[string]*int16) map[string]int16 {
return ptr.ToInt16Map(vs)
}
// ToInt32 returns int32 value dereferenced if the passed
// in pointer was not nil. Returns a int32 zero value if the
// pointer was nil.
func ToInt32(p *int32) (v int32) {
return ptr.ToInt32(p)
}
// ToInt32Slice returns a slice of int32 values, that are
// dereferenced if the passed in pointer was not nil. Returns a int32
// zero value if the pointer was nil.
func ToInt32Slice(vs []*int32) []int32 {
return ptr.ToInt32Slice(vs)
}
// ToInt32Map returns a map of int32 values, that are
// dereferenced if the passed in pointer was not nil. The int32
// zero value is used if the pointer was nil.
func ToInt32Map(vs map[string]*int32) map[string]int32 {
return ptr.ToInt32Map(vs)
}
// ToInt64 returns int64 value dereferenced if the passed
// in pointer was not nil. Returns a int64 zero value if the
// pointer was nil.
func ToInt64(p *int64) (v int64) {
return ptr.ToInt64(p)
}
// ToInt64Slice returns a slice of int64 values, that are
// dereferenced if the passed in pointer was not nil. Returns a int64
// zero value if the pointer was nil.
func ToInt64Slice(vs []*int64) []int64 {
return ptr.ToInt64Slice(vs)
}
// ToInt64Map returns a map of int64 values, that are
// dereferenced if the passed in pointer was not nil. The int64
// zero value is used if the pointer was nil.
func ToInt64Map(vs map[string]*int64) map[string]int64 {
return ptr.ToInt64Map(vs)
}
// ToUint returns uint value dereferenced if the passed
// in pointer was not nil. Returns a uint zero value if the
// pointer was nil.
func ToUint(p *uint) (v uint) {
return ptr.ToUint(p)
}
// ToUintSlice returns a slice of uint values, that are
// dereferenced if the passed in pointer was not nil. Returns a uint
// zero value if the pointer was nil.
func ToUintSlice(vs []*uint) []uint {
return ptr.ToUintSlice(vs)
}
// ToUintMap returns a map of uint values, that are
// dereferenced if the passed in pointer was not nil. The uint
// zero value is used if the pointer was nil.
func ToUintMap(vs map[string]*uint) map[string]uint {
return ptr.ToUintMap(vs)
}
// ToUint8 returns uint8 value dereferenced if the passed
// in pointer was not nil. Returns a uint8 zero value if the
// pointer was nil.
func ToUint8(p *uint8) (v uint8) {
return ptr.ToUint8(p)
}
// ToUint8Slice returns a slice of uint8 values, that are
// dereferenced if the passed in pointer was not nil. Returns a uint8
// zero value if the pointer was nil.
func ToUint8Slice(vs []*uint8) []uint8 {
return ptr.ToUint8Slice(vs)
}
// ToUint8Map returns a map of uint8 values, that are
// dereferenced if the passed in pointer was not nil. The uint8
// zero value is used if the pointer was nil.
func ToUint8Map(vs map[string]*uint8) map[string]uint8 {
return ptr.ToUint8Map(vs)
}
// ToUint16 returns uint16 value dereferenced if the passed
// in pointer was not nil. Returns a uint16 zero value if the
// pointer was nil.
func ToUint16(p *uint16) (v uint16) {
return ptr.ToUint16(p)
}
// ToUint16Slice returns a slice of uint16 values, that are
// dereferenced if the passed in pointer was not nil. Returns a uint16
// zero value if the pointer was nil.
func ToUint16Slice(vs []*uint16) []uint16 {
return ptr.ToUint16Slice(vs)
}
// ToUint16Map returns a map of uint16 values, that are
// dereferenced if the passed in pointer was not nil. The uint16
// zero value is used if the pointer was nil.
func ToUint16Map(vs map[string]*uint16) map[string]uint16 {
return ptr.ToUint16Map(vs)
}
// ToUint32 returns uint32 value dereferenced if the passed
// in pointer was not nil. Returns a uint32 zero value if the
// pointer was nil.
func ToUint32(p *uint32) (v uint32) {
return ptr.ToUint32(p)
}
// ToUint32Slice returns a slice of uint32 values, that are
// dereferenced if the passed in pointer was not nil. Returns a uint32
// zero value if the pointer was nil.
func ToUint32Slice(vs []*uint32) []uint32 {
return ptr.ToUint32Slice(vs)
}
// ToUint32Map returns a map of uint32 values, that are
// dereferenced if the passed in pointer was not nil. The uint32
// zero value is used if the pointer was nil.
func ToUint32Map(vs map[string]*uint32) map[string]uint32 {
return ptr.ToUint32Map(vs)
}
// ToUint64 returns uint64 value dereferenced if the passed
// in pointer was not nil. Returns a uint64 zero value if the
// pointer was nil.
func ToUint64(p *uint64) (v uint64) {
return ptr.ToUint64(p)
}
// ToUint64Slice returns a slice of uint64 values, that are
// dereferenced if the passed in pointer was not nil. Returns a uint64
// zero value if the pointer was nil.
func ToUint64Slice(vs []*uint64) []uint64 {
return ptr.ToUint64Slice(vs)
}
// ToUint64Map returns a map of uint64 values, that are
// dereferenced if the passed in pointer was not nil. The uint64
// zero value is used if the pointer was nil.
func ToUint64Map(vs map[string]*uint64) map[string]uint64 {
return ptr.ToUint64Map(vs)
}
// ToFloat32 returns float32 value dereferenced if the passed
// in pointer was not nil. Returns a float32 zero value if the
// pointer was nil.
func ToFloat32(p *float32) (v float32) {
return ptr.ToFloat32(p)
}
// ToFloat32Slice returns a slice of float32 values, that are
// dereferenced if the passed in pointer was not nil. Returns a float32
// zero value if the pointer was nil.
func ToFloat32Slice(vs []*float32) []float32 {
return ptr.ToFloat32Slice(vs)
}
// ToFloat32Map returns a map of float32 values, that are
// dereferenced if the passed in pointer was not nil. The float32
// zero value is used if the pointer was nil.
func ToFloat32Map(vs map[string]*float32) map[string]float32 {
return ptr.ToFloat32Map(vs)
}
// ToFloat64 returns float64 value dereferenced if the passed
// in pointer was not nil. Returns a float64 zero value if the
// pointer was nil.
func ToFloat64(p *float64) (v float64) {
return ptr.ToFloat64(p)
}
// ToFloat64Slice returns a slice of float64 values, that are
// dereferenced if the passed in pointer was not nil. Returns a float64
// zero value if the pointer was nil.
func ToFloat64Slice(vs []*float64) []float64 {
return ptr.ToFloat64Slice(vs)
}
// ToFloat64Map returns a map of float64 values, that are
// dereferenced if the passed in pointer was not nil. The float64
// zero value is used if the pointer was nil.
func ToFloat64Map(vs map[string]*float64) map[string]float64 {
return ptr.ToFloat64Map(vs)
}
// ToTime returns time.Time value dereferenced if the passed
// in pointer was not nil. Returns a time.Time zero value if the
// pointer was nil.
func ToTime(p *time.Time) (v time.Time) {
return ptr.ToTime(p)
}
// ToTimeSlice returns a slice of time.Time values, that are
// dereferenced if the passed in pointer was not nil. Returns a time.Time
// zero value if the pointer was nil.
func ToTimeSlice(vs []*time.Time) []time.Time {
return ptr.ToTimeSlice(vs)
}
// ToTimeMap returns a map of time.Time values, that are
// dereferenced if the passed in pointer was not nil. The time.Time
// zero value is used if the pointer was nil.
func ToTimeMap(vs map[string]*time.Time) map[string]time.Time {
return ptr.ToTimeMap(vs)
}
// ToDuration returns time.Duration value dereferenced if the passed
// in pointer was not nil. Returns a time.Duration zero value if the
// pointer was nil.
func ToDuration(p *time.Duration) (v time.Duration) {
return ptr.ToDuration(p)
}
// ToDurationSlice returns a slice of time.Duration values, that are
// dereferenced if the passed in pointer was not nil. Returns a time.Duration
// zero value if the pointer was nil.
func ToDurationSlice(vs []*time.Duration) []time.Duration {
return ptr.ToDurationSlice(vs)
}
// ToDurationMap returns a map of time.Duration values, that are
// dereferenced if the passed in pointer was not nil. The time.Duration
// zero value is used if the pointer was nil.
func ToDurationMap(vs map[string]*time.Duration) map[string]time.Duration {
return ptr.ToDurationMap(vs)
}
-6
View File
@@ -1,6 +0,0 @@
// Code generated by internal/repotools/cmd/updatemodulemeta DO NOT EDIT.
package aws
// goModuleVersion is the tagged release for this module
const goModuleVersion = "1.39.2"
-119
View File
@@ -1,119 +0,0 @@
// Code generated by aws/logging_generate.go DO NOT EDIT.
package aws
// ClientLogMode represents the logging mode of SDK clients. The client logging mode is a bit-field where
// each bit is a flag that describes the logging behavior for one or more client components.
// The entire 64-bit group is reserved for later expansion by the SDK.
//
// Example: Setting ClientLogMode to enable logging of retries and requests
//
// clientLogMode := aws.LogRetries | aws.LogRequest
//
// Example: Adding an additional log mode to an existing ClientLogMode value
//
// clientLogMode |= aws.LogResponse
type ClientLogMode uint64
// Supported ClientLogMode bits that can be configured to toggle logging of specific SDK events.
const (
LogSigning ClientLogMode = 1 << (64 - 1 - iota)
LogRetries
LogRequest
LogRequestWithBody
LogResponse
LogResponseWithBody
LogDeprecatedUsage
LogRequestEventMessage
LogResponseEventMessage
)
// IsSigning returns whether the Signing logging mode bit is set
func (m ClientLogMode) IsSigning() bool {
return m&LogSigning != 0
}
// IsRetries returns whether the Retries logging mode bit is set
func (m ClientLogMode) IsRetries() bool {
return m&LogRetries != 0
}
// IsRequest returns whether the Request logging mode bit is set
func (m ClientLogMode) IsRequest() bool {
return m&LogRequest != 0
}
// IsRequestWithBody returns whether the RequestWithBody logging mode bit is set
func (m ClientLogMode) IsRequestWithBody() bool {
return m&LogRequestWithBody != 0
}
// IsResponse returns whether the Response logging mode bit is set
func (m ClientLogMode) IsResponse() bool {
return m&LogResponse != 0
}
// IsResponseWithBody returns whether the ResponseWithBody logging mode bit is set
func (m ClientLogMode) IsResponseWithBody() bool {
return m&LogResponseWithBody != 0
}
// IsDeprecatedUsage returns whether the DeprecatedUsage logging mode bit is set
func (m ClientLogMode) IsDeprecatedUsage() bool {
return m&LogDeprecatedUsage != 0
}
// IsRequestEventMessage returns whether the RequestEventMessage logging mode bit is set
func (m ClientLogMode) IsRequestEventMessage() bool {
return m&LogRequestEventMessage != 0
}
// IsResponseEventMessage returns whether the ResponseEventMessage logging mode bit is set
func (m ClientLogMode) IsResponseEventMessage() bool {
return m&LogResponseEventMessage != 0
}
// ClearSigning clears the Signing logging mode bit
func (m *ClientLogMode) ClearSigning() {
*m &^= LogSigning
}
// ClearRetries clears the Retries logging mode bit
func (m *ClientLogMode) ClearRetries() {
*m &^= LogRetries
}
// ClearRequest clears the Request logging mode bit
func (m *ClientLogMode) ClearRequest() {
*m &^= LogRequest
}
// ClearRequestWithBody clears the RequestWithBody logging mode bit
func (m *ClientLogMode) ClearRequestWithBody() {
*m &^= LogRequestWithBody
}
// ClearResponse clears the Response logging mode bit
func (m *ClientLogMode) ClearResponse() {
*m &^= LogResponse
}
// ClearResponseWithBody clears the ResponseWithBody logging mode bit
func (m *ClientLogMode) ClearResponseWithBody() {
*m &^= LogResponseWithBody
}
// ClearDeprecatedUsage clears the DeprecatedUsage logging mode bit
func (m *ClientLogMode) ClearDeprecatedUsage() {
*m &^= LogDeprecatedUsage
}
// ClearRequestEventMessage clears the RequestEventMessage logging mode bit
func (m *ClientLogMode) ClearRequestEventMessage() {
*m &^= LogRequestEventMessage
}
// ClearResponseEventMessage clears the ResponseEventMessage logging mode bit
func (m *ClientLogMode) ClearResponseEventMessage() {
*m &^= LogResponseEventMessage
}
-95
View File
@@ -1,95 +0,0 @@
//go:build clientlogmode
// +build clientlogmode
package main
import (
"fmt"
"log"
"os"
"strings"
"text/template"
)
var config = struct {
ModeBits []string
}{
// Items should be appended only to keep bit-flag positions stable
ModeBits: []string{
"Signing",
"Retries",
"Request",
"RequestWithBody",
"Response",
"ResponseWithBody",
"DeprecatedUsage",
"RequestEventMessage",
"ResponseEventMessage",
},
}
func bitName(name string) string {
return strings.ToUpper(name[:1]) + name[1:]
}
var tmpl = template.Must(template.New("ClientLogMode").Funcs(map[string]interface{}{
"symbolName": func(name string) string {
return "Log" + bitName(name)
},
"bitName": bitName,
}).Parse(`// Code generated by aws/logging_generate.go DO NOT EDIT.
package aws
// ClientLogMode represents the logging mode of SDK clients. The client logging mode is a bit-field where
// each bit is a flag that describes the logging behavior for one or more client components.
// The entire 64-bit group is reserved for later expansion by the SDK.
//
// Example: Setting ClientLogMode to enable logging of retries and requests
// clientLogMode := aws.LogRetries | aws.LogRequest
//
// Example: Adding an additional log mode to an existing ClientLogMode value
// clientLogMode |= aws.LogResponse
type ClientLogMode uint64
// Supported ClientLogMode bits that can be configured to toggle logging of specific SDK events.
const (
{{- range $index, $field := .ModeBits }}
{{ (symbolName $field) }}{{- if (eq 0 $index) }} ClientLogMode = 1 << (64 - 1 - iota){{- end }}
{{- end }}
)
{{ range $_, $field := .ModeBits }}
// Is{{- bitName $field }} returns whether the {{ bitName $field }} logging mode bit is set
func (m ClientLogMode) Is{{- bitName $field }}() bool {
return m&{{- (symbolName $field) }} != 0
}
{{ end }}
{{- range $_, $field := .ModeBits }}
// Clear{{- bitName $field }} clears the {{ bitName $field }} logging mode bit
func (m *ClientLogMode) Clear{{- bitName $field }}() {
*m &^= {{ (symbolName $field) }}
}
{{ end -}}
`))
func main() {
uniqueBitFields := make(map[string]struct{})
for _, bitName := range config.ModeBits {
if _, ok := uniqueBitFields[strings.ToLower(bitName)]; ok {
panic(fmt.Sprintf("duplicate bit field: %s", bitName))
}
uniqueBitFields[bitName] = struct{}{}
}
file, err := os.Create("logging.go")
if err != nil {
log.Fatal(err)
}
defer file.Close()
err = tmpl.Execute(file, config)
if err != nil {
log.Fatal(err)
}
}
-213
View File
@@ -1,213 +0,0 @@
package middleware
import (
"context"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/smithy-go/middleware"
)
// RegisterServiceMetadata registers metadata about the service and operation into the middleware context
// so that it is available at runtime for other middleware to introspect.
type RegisterServiceMetadata struct {
ServiceID string
SigningName string
Region string
OperationName string
}
// ID returns the middleware identifier.
func (s *RegisterServiceMetadata) ID() string {
return "RegisterServiceMetadata"
}
// HandleInitialize registers service metadata information into the middleware context, allowing for introspection.
func (s RegisterServiceMetadata) HandleInitialize(
ctx context.Context, in middleware.InitializeInput, next middleware.InitializeHandler,
) (out middleware.InitializeOutput, metadata middleware.Metadata, err error) {
if len(s.ServiceID) > 0 {
ctx = SetServiceID(ctx, s.ServiceID)
}
if len(s.SigningName) > 0 {
ctx = SetSigningName(ctx, s.SigningName)
}
if len(s.Region) > 0 {
ctx = setRegion(ctx, s.Region)
}
if len(s.OperationName) > 0 {
ctx = setOperationName(ctx, s.OperationName)
}
return next.HandleInitialize(ctx, in)
}
// service metadata keys for storing and lookup of runtime stack information.
type (
serviceIDKey struct{}
signingNameKey struct{}
signingRegionKey struct{}
regionKey struct{}
operationNameKey struct{}
partitionIDKey struct{}
requiresLegacyEndpointsKey struct{}
)
// GetServiceID retrieves the service id from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func GetServiceID(ctx context.Context) (v string) {
v, _ = middleware.GetStackValue(ctx, serviceIDKey{}).(string)
return v
}
// GetSigningName retrieves the service signing name from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
//
// Deprecated: This value is unstable. The resolved signing name is available
// in the signer properties object passed to the signer.
func GetSigningName(ctx context.Context) (v string) {
v, _ = middleware.GetStackValue(ctx, signingNameKey{}).(string)
return v
}
// GetSigningRegion retrieves the region from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
//
// Deprecated: This value is unstable. The resolved signing region is available
// in the signer properties object passed to the signer.
func GetSigningRegion(ctx context.Context) (v string) {
v, _ = middleware.GetStackValue(ctx, signingRegionKey{}).(string)
return v
}
// GetRegion retrieves the endpoint region from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func GetRegion(ctx context.Context) (v string) {
v, _ = middleware.GetStackValue(ctx, regionKey{}).(string)
return v
}
// GetOperationName retrieves the service operation metadata from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func GetOperationName(ctx context.Context) (v string) {
v, _ = middleware.GetStackValue(ctx, operationNameKey{}).(string)
return v
}
// GetPartitionID retrieves the endpoint partition id from the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func GetPartitionID(ctx context.Context) string {
v, _ := middleware.GetStackValue(ctx, partitionIDKey{}).(string)
return v
}
// GetRequiresLegacyEndpoints the flag used to indicate if legacy endpoint
// customizations need to be executed.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func GetRequiresLegacyEndpoints(ctx context.Context) bool {
v, _ := middleware.GetStackValue(ctx, requiresLegacyEndpointsKey{}).(bool)
return v
}
// SetRequiresLegacyEndpoints set or modifies the flag indicated that
// legacy endpoint customizations are needed.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func SetRequiresLegacyEndpoints(ctx context.Context, value bool) context.Context {
return middleware.WithStackValue(ctx, requiresLegacyEndpointsKey{}, value)
}
// SetSigningName set or modifies the sigv4 or sigv4a signing name on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
//
// Deprecated: This value is unstable. Use WithSigV4SigningName client option
// funcs instead.
func SetSigningName(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, signingNameKey{}, value)
}
// SetSigningRegion sets or modifies the region on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
//
// Deprecated: This value is unstable. Use WithSigV4SigningRegion client option
// funcs instead.
func SetSigningRegion(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, signingRegionKey{}, value)
}
// SetServiceID sets the service id on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func SetServiceID(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, serviceIDKey{}, value)
}
// setRegion sets the endpoint region on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func setRegion(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, regionKey{}, value)
}
// setOperationName sets the service operation on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func setOperationName(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, operationNameKey{}, value)
}
// SetPartitionID sets the partition id of a resolved region on the context
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func SetPartitionID(ctx context.Context, value string) context.Context {
return middleware.WithStackValue(ctx, partitionIDKey{}, value)
}
// EndpointSource key
type endpointSourceKey struct{}
// GetEndpointSource returns an endpoint source if set on context
func GetEndpointSource(ctx context.Context) (v aws.EndpointSource) {
v, _ = middleware.GetStackValue(ctx, endpointSourceKey{}).(aws.EndpointSource)
return v
}
// SetEndpointSource sets endpoint source on context
func SetEndpointSource(ctx context.Context, value aws.EndpointSource) context.Context {
return middleware.WithStackValue(ctx, endpointSourceKey{}, value)
}
type signingCredentialsKey struct{}
// GetSigningCredentials returns the credentials that were used for signing if set on context.
func GetSigningCredentials(ctx context.Context) (v aws.Credentials) {
v, _ = middleware.GetStackValue(ctx, signingCredentialsKey{}).(aws.Credentials)
return v
}
// SetSigningCredentials sets the credentails used for signing on the context.
func SetSigningCredentials(ctx context.Context, value aws.Credentials) context.Context {
return middleware.WithStackValue(ctx, signingCredentialsKey{}, value)
}
-168
View File
@@ -1,168 +0,0 @@
package middleware
import (
"context"
"fmt"
"time"
"github.com/aws/aws-sdk-go-v2/internal/rand"
"github.com/aws/aws-sdk-go-v2/internal/sdk"
"github.com/aws/smithy-go/logging"
"github.com/aws/smithy-go/middleware"
smithyrand "github.com/aws/smithy-go/rand"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
// ClientRequestID is a Smithy BuildMiddleware that will generate a unique ID for logical API operation
// invocation.
type ClientRequestID struct{}
// ID the identifier for the ClientRequestID
func (r *ClientRequestID) ID() string {
return "ClientRequestID"
}
// HandleBuild attaches a unique operation invocation id for the operation to the request
func (r ClientRequestID) HandleBuild(ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler) (
out middleware.BuildOutput, metadata middleware.Metadata, err error,
) {
req, ok := in.Request.(*smithyhttp.Request)
if !ok {
return out, metadata, fmt.Errorf("unknown transport type %T", req)
}
invocationID, err := smithyrand.NewUUID(rand.Reader).GetUUID()
if err != nil {
return out, metadata, err
}
const invocationIDHeader = "Amz-Sdk-Invocation-Id"
req.Header[invocationIDHeader] = append(req.Header[invocationIDHeader][:0], invocationID)
return next.HandleBuild(ctx, in)
}
// RecordResponseTiming records the response timing for the SDK client requests.
type RecordResponseTiming struct{}
// ID is the middleware identifier
func (a *RecordResponseTiming) ID() string {
return "RecordResponseTiming"
}
// HandleDeserialize calculates response metadata and clock skew
func (a RecordResponseTiming) HandleDeserialize(ctx context.Context, in middleware.DeserializeInput, next middleware.DeserializeHandler) (
out middleware.DeserializeOutput, metadata middleware.Metadata, err error,
) {
out, metadata, err = next.HandleDeserialize(ctx, in)
responseAt := sdk.NowTime()
setResponseAt(&metadata, responseAt)
var serverTime time.Time
switch resp := out.RawResponse.(type) {
case *smithyhttp.Response:
respDateHeader := resp.Header.Get("Date")
if len(respDateHeader) == 0 {
break
}
var parseErr error
serverTime, parseErr = smithyhttp.ParseTime(respDateHeader)
if parseErr != nil {
logger := middleware.GetLogger(ctx)
logger.Logf(logging.Warn, "failed to parse response Date header value, got %v",
parseErr.Error())
break
}
setServerTime(&metadata, serverTime)
}
if !serverTime.IsZero() {
attemptSkew := serverTime.Sub(responseAt)
setAttemptSkew(&metadata, attemptSkew)
}
return out, metadata, err
}
type responseAtKey struct{}
// GetResponseAt returns the time response was received at.
func GetResponseAt(metadata middleware.Metadata) (v time.Time, ok bool) {
v, ok = metadata.Get(responseAtKey{}).(time.Time)
return v, ok
}
// setResponseAt sets the response time on the metadata.
func setResponseAt(metadata *middleware.Metadata, v time.Time) {
metadata.Set(responseAtKey{}, v)
}
type serverTimeKey struct{}
// GetServerTime returns the server time for response.
func GetServerTime(metadata middleware.Metadata) (v time.Time, ok bool) {
v, ok = metadata.Get(serverTimeKey{}).(time.Time)
return v, ok
}
// setServerTime sets the server time on the metadata.
func setServerTime(metadata *middleware.Metadata, v time.Time) {
metadata.Set(serverTimeKey{}, v)
}
type attemptSkewKey struct{}
// GetAttemptSkew returns Attempt clock skew for response from metadata.
func GetAttemptSkew(metadata middleware.Metadata) (v time.Duration, ok bool) {
v, ok = metadata.Get(attemptSkewKey{}).(time.Duration)
return v, ok
}
// setAttemptSkew sets the attempt clock skew on the metadata.
func setAttemptSkew(metadata *middleware.Metadata, v time.Duration) {
metadata.Set(attemptSkewKey{}, v)
}
// AddClientRequestIDMiddleware adds ClientRequestID to the middleware stack
func AddClientRequestIDMiddleware(stack *middleware.Stack) error {
return stack.Build.Add(&ClientRequestID{}, middleware.After)
}
// AddRecordResponseTiming adds RecordResponseTiming middleware to the
// middleware stack.
func AddRecordResponseTiming(stack *middleware.Stack) error {
return stack.Deserialize.Add(&RecordResponseTiming{}, middleware.After)
}
// rawResponseKey is the accessor key used to store and access the
// raw response within the response metadata.
type rawResponseKey struct{}
// AddRawResponse middleware adds raw response on to the metadata
type AddRawResponse struct{}
// ID the identifier for the ClientRequestID
func (m *AddRawResponse) ID() string {
return "AddRawResponseToMetadata"
}
// HandleDeserialize adds raw response on the middleware metadata
func (m AddRawResponse) HandleDeserialize(ctx context.Context, in middleware.DeserializeInput, next middleware.DeserializeHandler) (
out middleware.DeserializeOutput, metadata middleware.Metadata, err error,
) {
out, metadata, err = next.HandleDeserialize(ctx, in)
metadata.Set(rawResponseKey{}, out.RawResponse)
return out, metadata, err
}
// AddRawResponseToMetadata adds middleware to the middleware stack that
// store raw response on to the metadata.
func AddRawResponseToMetadata(stack *middleware.Stack) error {
return stack.Deserialize.Add(&AddRawResponse{}, middleware.Before)
}
// GetRawResponse returns raw response set on metadata
func GetRawResponse(metadata middleware.Metadata) interface{} {
return metadata.Get(rawResponseKey{})
}
-24
View File
@@ -1,24 +0,0 @@
//go:build go1.16
// +build go1.16
package middleware
import "runtime"
func getNormalizedOSName() (os string) {
switch runtime.GOOS {
case "android":
os = "android"
case "linux":
os = "linux"
case "windows":
os = "windows"
case "darwin":
os = "macos"
case "ios":
os = "ios"
default:
os = "other"
}
return os
}
-24
View File
@@ -1,24 +0,0 @@
//go:build !go1.16
// +build !go1.16
package middleware
import "runtime"
func getNormalizedOSName() (os string) {
switch runtime.GOOS {
case "android":
os = "android"
case "linux":
os = "linux"
case "windows":
os = "windows"
case "darwin":
// Due to Apple M1 we can't distinguish between macOS and iOS when GOOS/GOARCH is darwin/amd64
// For now declare this as "other" until we have a better detection mechanism.
fallthrough
default:
os = "other"
}
return os
}
@@ -1,94 +0,0 @@
package middleware
import (
"context"
"fmt"
"github.com/aws/smithy-go/middleware"
smithyhttp "github.com/aws/smithy-go/transport/http"
"os"
)
const envAwsLambdaFunctionName = "AWS_LAMBDA_FUNCTION_NAME"
const envAmznTraceID = "_X_AMZN_TRACE_ID"
const amznTraceIDHeader = "X-Amzn-Trace-Id"
// AddRecursionDetection adds recursionDetection to the middleware stack
func AddRecursionDetection(stack *middleware.Stack) error {
return stack.Build.Add(&RecursionDetection{}, middleware.After)
}
// RecursionDetection detects Lambda environment and sets its X-Ray trace ID to request header if absent
// to avoid recursion invocation in Lambda
type RecursionDetection struct{}
// ID returns the middleware identifier
func (m *RecursionDetection) ID() string {
return "RecursionDetection"
}
// HandleBuild detects Lambda environment and adds its trace ID to request header if absent
func (m *RecursionDetection) HandleBuild(
ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler,
) (
out middleware.BuildOutput, metadata middleware.Metadata, err error,
) {
req, ok := in.Request.(*smithyhttp.Request)
if !ok {
return out, metadata, fmt.Errorf("unknown request type %T", req)
}
_, hasLambdaEnv := os.LookupEnv(envAwsLambdaFunctionName)
xAmznTraceID, hasTraceID := os.LookupEnv(envAmznTraceID)
value := req.Header.Get(amznTraceIDHeader)
// only set the X-Amzn-Trace-Id header when it is not set initially, the
// current environment is Lambda and the _X_AMZN_TRACE_ID env variable exists
if value != "" || !hasLambdaEnv || !hasTraceID {
return next.HandleBuild(ctx, in)
}
req.Header.Set(amznTraceIDHeader, percentEncode(xAmznTraceID))
return next.HandleBuild(ctx, in)
}
func percentEncode(s string) string {
upperhex := "0123456789ABCDEF"
hexCount := 0
for i := 0; i < len(s); i++ {
c := s[i]
if shouldEncode(c) {
hexCount++
}
}
if hexCount == 0 {
return s
}
required := len(s) + 2*hexCount
t := make([]byte, required)
j := 0
for i := 0; i < len(s); i++ {
if c := s[i]; shouldEncode(c) {
t[j] = '%'
t[j+1] = upperhex[c>>4]
t[j+2] = upperhex[c&15]
j += 3
} else {
t[j] = c
j++
}
}
return string(t)
}
func shouldEncode(c byte) bool {
if 'a' <= c && c <= 'z' || 'A' <= c && c <= 'Z' || '0' <= c && c <= '9' {
return false
}
switch c {
case '-', '=', ';', ':', '+', '&', '[', ']', '{', '}', '"', '\'', ',':
return false
default:
return true
}
}
-27
View File
@@ -1,27 +0,0 @@
package middleware
import (
"github.com/aws/smithy-go/middleware"
)
// requestIDKey is used to retrieve request id from response metadata
type requestIDKey struct{}
// SetRequestIDMetadata sets the provided request id over middleware metadata
func SetRequestIDMetadata(metadata *middleware.Metadata, id string) {
metadata.Set(requestIDKey{}, id)
}
// GetRequestIDMetadata retrieves the request id from middleware metadata
// returns string and bool indicating value of request id, whether request id was set.
func GetRequestIDMetadata(metadata middleware.Metadata) (string, bool) {
if !metadata.Has(requestIDKey{}) {
return "", false
}
v, ok := metadata.Get(requestIDKey{}).(string)
if !ok {
return "", true
}
return v, true
}
@@ -1,57 +0,0 @@
package middleware
import (
"context"
"github.com/aws/smithy-go/middleware"
"github.com/aws/smithy-go/tracing"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
// AddRequestIDRetrieverMiddleware adds request id retriever middleware
func AddRequestIDRetrieverMiddleware(stack *middleware.Stack) error {
// add error wrapper middleware before operation deserializers so that it can wrap the error response
// returned by operation deserializers
return stack.Deserialize.Insert(&RequestIDRetriever{}, "OperationDeserializer", middleware.Before)
}
// RequestIDRetriever middleware captures the AWS service request ID from the
// raw response.
type RequestIDRetriever struct {
}
// ID returns the middleware identifier
func (m *RequestIDRetriever) ID() string {
return "RequestIDRetriever"
}
// HandleDeserialize pulls the AWS request ID from the response, storing it in
// operation metadata.
func (m *RequestIDRetriever) HandleDeserialize(ctx context.Context, in middleware.DeserializeInput, next middleware.DeserializeHandler) (
out middleware.DeserializeOutput, metadata middleware.Metadata, err error,
) {
out, metadata, err = next.HandleDeserialize(ctx, in)
resp, ok := out.RawResponse.(*smithyhttp.Response)
if !ok {
// No raw response to wrap with.
return out, metadata, err
}
// Different header which can map to request id
requestIDHeaderList := []string{"X-Amzn-Requestid", "X-Amz-RequestId"}
for _, h := range requestIDHeaderList {
// check for headers known to contain Request id
if v := resp.Header.Get(h); len(v) != 0 {
// set reqID on metadata for successful responses.
SetRequestIDMetadata(&metadata, v)
span, _ := tracing.GetSpan(ctx)
span.SetProperty("aws.request_id", v)
break
}
}
return out, metadata, err
}
-393
View File
@@ -1,393 +0,0 @@
package middleware
import (
"context"
"fmt"
"os"
"runtime"
"sort"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/smithy-go/middleware"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
var languageVersion = strings.TrimPrefix(runtime.Version(), "go")
// SDKAgentKeyType is the metadata type to add to the SDK agent string
type SDKAgentKeyType int
// The set of valid SDKAgentKeyType constants. If an unknown value is assigned for SDKAgentKeyType it will
// be mapped to AdditionalMetadata.
const (
_ SDKAgentKeyType = iota
APIMetadata
OperatingSystemMetadata
LanguageMetadata
EnvironmentMetadata
FeatureMetadata
ConfigMetadata
FrameworkMetadata
AdditionalMetadata
ApplicationIdentifier
FeatureMetadata2
)
// Hardcoded value to specify which version of the user agent we're using
const uaMetadata = "ua/2.1"
func (k SDKAgentKeyType) string() string {
switch k {
case APIMetadata:
return "api"
case OperatingSystemMetadata:
return "os"
case LanguageMetadata:
return "lang"
case EnvironmentMetadata:
return "exec-env"
case FeatureMetadata:
return "ft"
case ConfigMetadata:
return "cfg"
case FrameworkMetadata:
return "lib"
case ApplicationIdentifier:
return "app"
case FeatureMetadata2:
return "m"
case AdditionalMetadata:
fallthrough
default:
return "md"
}
}
const execEnvVar = `AWS_EXECUTION_ENV`
var validChars = map[rune]bool{
'!': true, '#': true, '$': true, '%': true, '&': true, '\'': true, '*': true, '+': true,
'-': true, '.': true, '^': true, '_': true, '`': true, '|': true, '~': true,
}
// UserAgentFeature enumerates tracked SDK features.
type UserAgentFeature string
// Enumerates UserAgentFeature.
const (
UserAgentFeatureResourceModel UserAgentFeature = "A" // n/a (we don't generate separate resource types)
UserAgentFeatureWaiter = "B"
UserAgentFeaturePaginator = "C"
UserAgentFeatureRetryModeLegacy = "D" // n/a (equivalent to standard)
UserAgentFeatureRetryModeStandard = "E"
UserAgentFeatureRetryModeAdaptive = "F"
UserAgentFeatureS3Transfer = "G"
UserAgentFeatureS3CryptoV1N = "H" // n/a (crypto client is external)
UserAgentFeatureS3CryptoV2 = "I" // n/a
UserAgentFeatureS3ExpressBucket = "J"
UserAgentFeatureS3AccessGrants = "K" // not yet implemented
UserAgentFeatureGZIPRequestCompression = "L"
UserAgentFeatureProtocolRPCV2CBOR = "M"
UserAgentFeatureAccountIDEndpoint = "O" // DO NOT IMPLEMENT: rules output is not currently defined. SDKs should not parse endpoints for feature information.
UserAgentFeatureAccountIDModePreferred = "P"
UserAgentFeatureAccountIDModeDisabled = "Q"
UserAgentFeatureAccountIDModeRequired = "R"
UserAgentFeatureRequestChecksumCRC32 = "U"
UserAgentFeatureRequestChecksumCRC32C = "V"
UserAgentFeatureRequestChecksumCRC64 = "W"
UserAgentFeatureRequestChecksumSHA1 = "X"
UserAgentFeatureRequestChecksumSHA256 = "Y"
UserAgentFeatureRequestChecksumWhenSupported = "Z"
UserAgentFeatureRequestChecksumWhenRequired = "a"
UserAgentFeatureResponseChecksumWhenSupported = "b"
UserAgentFeatureResponseChecksumWhenRequired = "c"
UserAgentFeatureDynamoDBUserAgent = "d" // not yet implemented
UserAgentFeatureCredentialsCode = "e"
UserAgentFeatureCredentialsJvmSystemProperties = "f" // n/a (this is not a JVM sdk)
UserAgentFeatureCredentialsEnvVars = "g"
UserAgentFeatureCredentialsEnvVarsStsWebIDToken = "h"
UserAgentFeatureCredentialsStsAssumeRole = "i"
UserAgentFeatureCredentialsStsAssumeRoleSaml = "j" // not yet implemented
UserAgentFeatureCredentialsStsAssumeRoleWebID = "k"
UserAgentFeatureCredentialsStsFederationToken = "l" // not yet implemented
UserAgentFeatureCredentialsStsSessionToken = "m" // not yet implemented
UserAgentFeatureCredentialsProfile = "n"
UserAgentFeatureCredentialsProfileSourceProfile = "o"
UserAgentFeatureCredentialsProfileNamedProvider = "p"
UserAgentFeatureCredentialsProfileStsWebIDToken = "q"
UserAgentFeatureCredentialsProfileSso = "r"
UserAgentFeatureCredentialsSso = "s"
UserAgentFeatureCredentialsProfileSsoLegacy = "t"
UserAgentFeatureCredentialsSsoLegacy = "u"
UserAgentFeatureCredentialsProfileProcess = "v"
UserAgentFeatureCredentialsProcess = "w"
UserAgentFeatureCredentialsBoto2ConfigFile = "x" // n/a (this is not boto/Python)
UserAgentFeatureCredentialsAwsSdkStore = "y" // n/a (this is used by .NET based sdk)
UserAgentFeatureCredentialsHTTP = "z"
UserAgentFeatureCredentialsIMDS = "0"
UserAgentFeatureBearerServiceEnvVars = "3"
)
var credentialSourceToFeature = map[aws.CredentialSource]UserAgentFeature{
aws.CredentialSourceCode: UserAgentFeatureCredentialsCode,
aws.CredentialSourceEnvVars: UserAgentFeatureCredentialsEnvVars,
aws.CredentialSourceEnvVarsSTSWebIDToken: UserAgentFeatureCredentialsEnvVarsStsWebIDToken,
aws.CredentialSourceSTSAssumeRole: UserAgentFeatureCredentialsStsAssumeRole,
aws.CredentialSourceSTSAssumeRoleSaml: UserAgentFeatureCredentialsStsAssumeRoleSaml,
aws.CredentialSourceSTSAssumeRoleWebID: UserAgentFeatureCredentialsStsAssumeRoleWebID,
aws.CredentialSourceSTSFederationToken: UserAgentFeatureCredentialsStsFederationToken,
aws.CredentialSourceSTSSessionToken: UserAgentFeatureCredentialsStsSessionToken,
aws.CredentialSourceProfile: UserAgentFeatureCredentialsProfile,
aws.CredentialSourceProfileSourceProfile: UserAgentFeatureCredentialsProfileSourceProfile,
aws.CredentialSourceProfileNamedProvider: UserAgentFeatureCredentialsProfileNamedProvider,
aws.CredentialSourceProfileSTSWebIDToken: UserAgentFeatureCredentialsProfileStsWebIDToken,
aws.CredentialSourceProfileSSO: UserAgentFeatureCredentialsProfileSso,
aws.CredentialSourceSSO: UserAgentFeatureCredentialsSso,
aws.CredentialSourceProfileSSOLegacy: UserAgentFeatureCredentialsProfileSsoLegacy,
aws.CredentialSourceSSOLegacy: UserAgentFeatureCredentialsSsoLegacy,
aws.CredentialSourceProfileProcess: UserAgentFeatureCredentialsProfileProcess,
aws.CredentialSourceProcess: UserAgentFeatureCredentialsProcess,
aws.CredentialSourceHTTP: UserAgentFeatureCredentialsHTTP,
aws.CredentialSourceIMDS: UserAgentFeatureCredentialsIMDS,
}
// RequestUserAgent is a build middleware that set the User-Agent for the request.
type RequestUserAgent struct {
sdkAgent, userAgent *smithyhttp.UserAgentBuilder
features map[UserAgentFeature]struct{}
}
// NewRequestUserAgent returns a new requestUserAgent which will set the User-Agent and X-Amz-User-Agent for the
// request.
//
// User-Agent example:
//
// aws-sdk-go-v2/1.2.3
//
// X-Amz-User-Agent example:
//
// aws-sdk-go-v2/1.2.3 md/GOOS/linux md/GOARCH/amd64 lang/go/1.15
func NewRequestUserAgent() *RequestUserAgent {
userAgent, sdkAgent := smithyhttp.NewUserAgentBuilder(), smithyhttp.NewUserAgentBuilder()
addProductName(userAgent)
addUserAgentMetadata(userAgent)
addProductName(sdkAgent)
r := &RequestUserAgent{
sdkAgent: sdkAgent,
userAgent: userAgent,
features: map[UserAgentFeature]struct{}{},
}
addSDKMetadata(r)
return r
}
func addSDKMetadata(r *RequestUserAgent) {
r.AddSDKAgentKey(OperatingSystemMetadata, getNormalizedOSName())
r.AddSDKAgentKeyValue(LanguageMetadata, "go", languageVersion)
r.AddSDKAgentKeyValue(AdditionalMetadata, "GOOS", runtime.GOOS)
r.AddSDKAgentKeyValue(AdditionalMetadata, "GOARCH", runtime.GOARCH)
if ev := os.Getenv(execEnvVar); len(ev) > 0 {
r.AddSDKAgentKey(EnvironmentMetadata, ev)
}
}
func addProductName(builder *smithyhttp.UserAgentBuilder) {
builder.AddKeyValue(aws.SDKName, aws.SDKVersion)
}
func addUserAgentMetadata(builder *smithyhttp.UserAgentBuilder) {
builder.AddKey(uaMetadata)
}
// AddUserAgentKey retrieves a requestUserAgent from the provided stack, or initializes one.
func AddUserAgentKey(key string) func(*middleware.Stack) error {
return func(stack *middleware.Stack) error {
requestUserAgent, err := getOrAddRequestUserAgent(stack)
if err != nil {
return err
}
requestUserAgent.AddUserAgentKey(key)
return nil
}
}
// AddUserAgentKeyValue retrieves a requestUserAgent from the provided stack, or initializes one.
func AddUserAgentKeyValue(key, value string) func(*middleware.Stack) error {
return func(stack *middleware.Stack) error {
requestUserAgent, err := getOrAddRequestUserAgent(stack)
if err != nil {
return err
}
requestUserAgent.AddUserAgentKeyValue(key, value)
return nil
}
}
// AddSDKAgentKey retrieves a requestUserAgent from the provided stack, or initializes one.
func AddSDKAgentKey(keyType SDKAgentKeyType, key string) func(*middleware.Stack) error {
return func(stack *middleware.Stack) error {
requestUserAgent, err := getOrAddRequestUserAgent(stack)
if err != nil {
return err
}
requestUserAgent.AddSDKAgentKey(keyType, key)
return nil
}
}
// AddSDKAgentKeyValue retrieves a requestUserAgent from the provided stack, or initializes one.
func AddSDKAgentKeyValue(keyType SDKAgentKeyType, key, value string) func(*middleware.Stack) error {
return func(stack *middleware.Stack) error {
requestUserAgent, err := getOrAddRequestUserAgent(stack)
if err != nil {
return err
}
requestUserAgent.AddSDKAgentKeyValue(keyType, key, value)
return nil
}
}
// AddRequestUserAgentMiddleware registers a requestUserAgent middleware on the stack if not present.
func AddRequestUserAgentMiddleware(stack *middleware.Stack) error {
_, err := getOrAddRequestUserAgent(stack)
return err
}
func getOrAddRequestUserAgent(stack *middleware.Stack) (*RequestUserAgent, error) {
id := (*RequestUserAgent)(nil).ID()
bm, ok := stack.Build.Get(id)
if !ok {
bm = NewRequestUserAgent()
err := stack.Build.Add(bm, middleware.After)
if err != nil {
return nil, err
}
}
requestUserAgent, ok := bm.(*RequestUserAgent)
if !ok {
return nil, fmt.Errorf("%T for %s middleware did not match expected type", bm, id)
}
return requestUserAgent, nil
}
// AddUserAgentKey adds the component identified by name to the User-Agent string.
func (u *RequestUserAgent) AddUserAgentKey(key string) {
u.userAgent.AddKey(strings.Map(rules, key))
}
// AddUserAgentKeyValue adds the key identified by the given name and value to the User-Agent string.
func (u *RequestUserAgent) AddUserAgentKeyValue(key, value string) {
u.userAgent.AddKeyValue(strings.Map(rules, key), strings.Map(rules, value))
}
// AddUserAgentFeature adds the feature ID to the tracking list to be emitted
// in the final User-Agent string.
func (u *RequestUserAgent) AddUserAgentFeature(feature UserAgentFeature) {
u.features[feature] = struct{}{}
}
// AddSDKAgentKey adds the component identified by name to the User-Agent string.
func (u *RequestUserAgent) AddSDKAgentKey(keyType SDKAgentKeyType, key string) {
// TODO: should target sdkAgent
u.userAgent.AddKey(keyType.string() + "/" + strings.Map(rules, key))
}
// AddSDKAgentKeyValue adds the key identified by the given name and value to the User-Agent string.
func (u *RequestUserAgent) AddSDKAgentKeyValue(keyType SDKAgentKeyType, key, value string) {
// TODO: should target sdkAgent
u.userAgent.AddKeyValue(keyType.string(), strings.Map(rules, key)+"#"+strings.Map(rules, value))
}
// AddCredentialsSource adds the credential source as a feature on the User-Agent string
func (u *RequestUserAgent) AddCredentialsSource(source aws.CredentialSource) {
x, ok := credentialSourceToFeature[source]
if ok {
u.AddUserAgentFeature(x)
}
}
// ID the name of the middleware.
func (u *RequestUserAgent) ID() string {
return "UserAgent"
}
// HandleBuild adds or appends the constructed user agent to the request.
func (u *RequestUserAgent) HandleBuild(ctx context.Context, in middleware.BuildInput, next middleware.BuildHandler) (
out middleware.BuildOutput, metadata middleware.Metadata, err error,
) {
switch req := in.Request.(type) {
case *smithyhttp.Request:
u.addHTTPUserAgent(req)
// TODO: To be re-enabled
// u.addHTTPSDKAgent(req)
default:
return out, metadata, fmt.Errorf("unknown transport type %T", in)
}
return next.HandleBuild(ctx, in)
}
func (u *RequestUserAgent) addHTTPUserAgent(request *smithyhttp.Request) {
const userAgent = "User-Agent"
if len(u.features) > 0 {
updateHTTPHeader(request, userAgent, buildFeatureMetrics(u.features))
}
updateHTTPHeader(request, userAgent, u.userAgent.Build())
}
func (u *RequestUserAgent) addHTTPSDKAgent(request *smithyhttp.Request) {
const sdkAgent = "X-Amz-User-Agent"
updateHTTPHeader(request, sdkAgent, u.sdkAgent.Build())
}
func updateHTTPHeader(request *smithyhttp.Request, header string, value string) {
var current string
if v := request.Header[header]; len(v) > 0 {
current = v[0]
}
if len(current) > 0 {
current = value + " " + current
} else {
current = value
}
request.Header[header] = append(request.Header[header][:0], current)
}
func rules(r rune) rune {
switch {
case r >= '0' && r <= '9':
return r
case r >= 'A' && r <= 'Z' || r >= 'a' && r <= 'z':
return r
case validChars[r]:
return r
default:
return '-'
}
}
func buildFeatureMetrics(features map[UserAgentFeature]struct{}) string {
fs := make([]string, 0, len(features))
for f := range features {
fs = append(fs, string(f))
}
sort.Strings(fs)
return fmt.Sprintf("%s/%s", FeatureMetadata2.string(), strings.Join(fs, ","))
}
-61
View File
@@ -1,61 +0,0 @@
package query
import (
"net/url"
"strconv"
)
// Array represents the encoding of Query lists and sets. A Query array is a
// representation of a list of values of a fixed type. A serialized array might
// look like the following:
//
// ListName.member.1=foo
// &ListName.member.2=bar
// &Listname.member.3=baz
type Array struct {
// The query values to add the array to.
values url.Values
// The array's prefix, which includes the names of all parent structures
// and ends with the name of the list. For example, the prefix might be
// "ParentStructure.ListName". This prefix will be used to form the full
// keys for each element in the list. For example, an entry might have the
// key "ParentStructure.ListName.member.MemberName.1".
//
// When the array is not flat the prefix will contain the memberName otherwise the memberName is ignored
prefix string
// Elements are stored in values, so we keep track of the list size here.
size int32
// Empty lists are encoded as "<prefix>=", if we add a value later we will
// remove this encoding
emptyValue Value
}
func newArray(values url.Values, prefix string, flat bool, memberName string) *Array {
emptyValue := newValue(values, prefix, flat)
emptyValue.String("")
if !flat {
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
prefix = prefix + keySeparator + memberName
}
return &Array{
values: values,
prefix: prefix,
emptyValue: emptyValue,
}
}
// Value adds a new element to the Query Array. Returns a Value type used to
// encode the array element.
func (a *Array) Value() Value {
if a.size == 0 {
delete(a.values, a.emptyValue.key)
}
// Query lists start a 1, so adjust the size first
a.size++
// Lists can't have flat members
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newValue(a.values, a.prefix+keySeparator+strconv.FormatInt(int64(a.size), 10), false)
}
-80
View File
@@ -1,80 +0,0 @@
package query
import (
"io"
"net/url"
"sort"
)
// Encoder is a Query encoder that supports construction of Query body
// values using methods.
type Encoder struct {
// The query values that will be built up to manage encoding.
values url.Values
// The writer that the encoded body will be written to.
writer io.Writer
Value
}
// NewEncoder returns a new Query body encoder
func NewEncoder(writer io.Writer) *Encoder {
values := url.Values{}
return &Encoder{
values: values,
writer: writer,
Value: newBaseValue(values),
}
}
// Encode returns the []byte slice representing the current
// state of the Query encoder.
func (e Encoder) Encode() error {
ws, ok := e.writer.(interface{ WriteString(string) (int, error) })
if !ok {
// Fall back to less optimal byte slice casting if WriteString isn't available.
ws = &wrapWriteString{writer: e.writer}
}
// Get the keys and sort them to have a stable output
keys := make([]string, 0, len(e.values))
for k := range e.values {
keys = append(keys, k)
}
sort.Strings(keys)
isFirstEntry := true
for _, key := range keys {
queryValues := e.values[key]
escapedKey := url.QueryEscape(key)
for _, value := range queryValues {
if !isFirstEntry {
if _, err := ws.WriteString(`&`); err != nil {
return err
}
} else {
isFirstEntry = false
}
if _, err := ws.WriteString(escapedKey); err != nil {
return err
}
if _, err := ws.WriteString(`=`); err != nil {
return err
}
if _, err := ws.WriteString(url.QueryEscape(value)); err != nil {
return err
}
}
}
return nil
}
// wrapWriteString wraps an io.Writer to provide a WriteString method
// where one is not available.
type wrapWriteString struct {
writer io.Writer
}
// WriteString writes a string to the wrapped writer by casting it to
// a byte array first.
func (w wrapWriteString) WriteString(v string) (int, error) {
return w.writer.Write([]byte(v))
}
-78
View File
@@ -1,78 +0,0 @@
package query
import (
"fmt"
"net/url"
)
// Map represents the encoding of Query maps. A Query map is a representation
// of a mapping of arbitrary string keys to arbitrary values of a fixed type.
// A Map differs from an Object in that the set of keys is not fixed, in that
// the values must all be of the same type, and that map entries are ordered.
// A serialized map might look like the following:
//
// MapName.entry.1.key=Foo
// &MapName.entry.1.value=spam
// &MapName.entry.2.key=Bar
// &MapName.entry.2.value=eggs
type Map struct {
// The query values to add the map to.
values url.Values
// The map's prefix, which includes the names of all parent structures
// and ends with the name of the object. For example, the prefix might be
// "ParentStructure.MapName". This prefix will be used to form the full
// keys for each key-value pair of the map. For example, a value might have
// the key "ParentStructure.MapName.1.value".
//
// While this is currently represented as a string that gets added to, it
// could also be represented as a stack that only gets condensed into a
// string when a finalized key is created. This could potentially reduce
// allocations.
prefix string
// Whether the map is flat or not. A map that is not flat will produce the
// following entries to the url.Values for a given key-value pair:
// MapName.entry.1.KeyLocationName=mykey
// MapName.entry.1.ValueLocationName=myvalue
// A map that is flat will produce the following:
// MapName.1.KeyLocationName=mykey
// MapName.1.ValueLocationName=myvalue
flat bool
// The location name of the key. In most cases this should be "key".
keyLocationName string
// The location name of the value. In most cases this should be "value".
valueLocationName string
// Elements are stored in values, so we keep track of the list size here.
size int32
}
func newMap(values url.Values, prefix string, flat bool, keyLocationName string, valueLocationName string) *Map {
return &Map{
values: values,
prefix: prefix,
flat: flat,
keyLocationName: keyLocationName,
valueLocationName: valueLocationName,
}
}
// Key adds the given named key to the Query map.
// Returns a Value encoder that should be used to encode a Query value type.
func (m *Map) Key(name string) Value {
// Query lists start a 1, so adjust the size first
m.size++
var key string
var value string
if m.flat {
key = fmt.Sprintf("%s.%d.%s", m.prefix, m.size, m.keyLocationName)
value = fmt.Sprintf("%s.%d.%s", m.prefix, m.size, m.valueLocationName)
} else {
key = fmt.Sprintf("%s.entry.%d.%s", m.prefix, m.size, m.keyLocationName)
value = fmt.Sprintf("%s.entry.%d.%s", m.prefix, m.size, m.valueLocationName)
}
// The key can only be a string, so we just go ahead and set it here
newValue(m.values, key, false).String(name)
// Maps can't have flat members
return newValue(m.values, value, false)
}
-62
View File
@@ -1,62 +0,0 @@
package query
import (
"context"
"fmt"
"io/ioutil"
"github.com/aws/smithy-go/middleware"
smithyhttp "github.com/aws/smithy-go/transport/http"
)
// AddAsGetRequestMiddleware adds a middleware to the Serialize stack after the
// operation serializer that will convert the query request body to a GET
// operation with the query message in the HTTP request querystring.
func AddAsGetRequestMiddleware(stack *middleware.Stack) error {
return stack.Serialize.Insert(&asGetRequest{}, "OperationSerializer", middleware.After)
}
type asGetRequest struct{}
func (*asGetRequest) ID() string { return "Query:AsGetRequest" }
func (m *asGetRequest) HandleSerialize(
ctx context.Context, input middleware.SerializeInput, next middleware.SerializeHandler,
) (
out middleware.SerializeOutput, metadata middleware.Metadata, err error,
) {
req, ok := input.Request.(*smithyhttp.Request)
if !ok {
return out, metadata, fmt.Errorf("expect smithy HTTP Request, got %T", input.Request)
}
req.Method = "GET"
// If the stream is not set, nothing else to do.
stream := req.GetStream()
if stream == nil {
return next.HandleSerialize(ctx, input)
}
// Clear the stream since there will not be any body.
req.Header.Del("Content-Type")
req, err = req.SetStream(nil)
if err != nil {
return out, metadata, fmt.Errorf("unable update request body %w", err)
}
input.Request = req
// Update request query with the body's query string value.
delim := ""
if len(req.URL.RawQuery) != 0 {
delim = "&"
}
b, err := ioutil.ReadAll(stream)
if err != nil {
return out, metadata, fmt.Errorf("unable to get request body %w", err)
}
req.URL.RawQuery += delim + string(b)
return next.HandleSerialize(ctx, input)
}
-68
View File
@@ -1,68 +0,0 @@
package query
import "net/url"
// Object represents the encoding of Query structures and unions. A Query
// object is a representation of a mapping of string keys to arbitrary
// values where there is a fixed set of keys whose values each have their
// own known type. A serialized object might look like the following:
//
// ObjectName.Foo=value
// &ObjectName.Bar=5
type Object struct {
// The query values to add the object to.
values url.Values
// The object's prefix, which includes the names of all parent structures
// and ends with the name of the object. For example, the prefix might be
// "ParentStructure.ObjectName". This prefix will be used to form the full
// keys for each member of the object. For example, a member might have the
// key "ParentStructure.ObjectName.MemberName".
//
// While this is currently represented as a string that gets added to, it
// could also be represented as a stack that only gets condensed into a
// string when a finalized key is created. This could potentially reduce
// allocations.
prefix string
}
func newObject(values url.Values, prefix string) *Object {
return &Object{
values: values,
prefix: prefix,
}
}
// Key adds the given named key to the Query object.
// Returns a Value encoder that should be used to encode a Query value type.
func (o *Object) Key(name string) Value {
return o.key(name, false)
}
// KeyWithValues adds the given named key to the Query object.
// Returns a Value encoder that should be used to encode a Query list of values.
func (o *Object) KeyWithValues(name string) Value {
return o.keyWithValues(name, false)
}
// FlatKey adds the given named key to the Query object.
// Returns a Value encoder that should be used to encode a Query value type. The
// value will be flattened if it is a map or array.
func (o *Object) FlatKey(name string) Value {
return o.key(name, true)
}
func (o *Object) key(name string, flatValue bool) Value {
if o.prefix != "" {
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newValue(o.values, o.prefix+keySeparator+name, flatValue)
}
return newValue(o.values, name, flatValue)
}
func (o *Object) keyWithValues(name string, flatValue bool) Value {
if o.prefix != "" {
// This uses string concatenation in place of fmt.Sprintf as fmt.Sprintf has a much higher resource overhead
return newAppendValue(o.values, o.prefix+keySeparator+name, flatValue)
}
return newAppendValue(o.values, name, flatValue)
}
-117
View File
@@ -1,117 +0,0 @@
package query
import (
"math/big"
"net/url"
"github.com/aws/smithy-go/encoding/httpbinding"
)
const keySeparator = "."
// Value represents a Query Value type.
type Value struct {
// The query values to add the value to.
values url.Values
// The value's key, which will form the prefix for complex types.
key string
// Whether the value should be flattened or not if it's a flattenable type.
flat bool
queryValue httpbinding.QueryValue
}
func newValue(values url.Values, key string, flat bool) Value {
return Value{
values: values,
key: key,
flat: flat,
queryValue: httpbinding.NewQueryValue(values, key, false),
}
}
func newAppendValue(values url.Values, key string, flat bool) Value {
return Value{
values: values,
key: key,
flat: flat,
queryValue: httpbinding.NewQueryValue(values, key, true),
}
}
func newBaseValue(values url.Values) Value {
return Value{
values: values,
queryValue: httpbinding.NewQueryValue(nil, "", false),
}
}
// Array returns a new Array encoder.
func (qv Value) Array(locationName string) *Array {
return newArray(qv.values, qv.key, qv.flat, locationName)
}
// Object returns a new Object encoder.
func (qv Value) Object() *Object {
return newObject(qv.values, qv.key)
}
// Map returns a new Map encoder.
func (qv Value) Map(keyLocationName string, valueLocationName string) *Map {
return newMap(qv.values, qv.key, qv.flat, keyLocationName, valueLocationName)
}
// Base64EncodeBytes encodes v as a base64 query string value.
// This is intended to enable compatibility with the JSON encoder.
func (qv Value) Base64EncodeBytes(v []byte) {
qv.queryValue.Blob(v)
}
// Boolean encodes v as a query string value
func (qv Value) Boolean(v bool) {
qv.queryValue.Boolean(v)
}
// String encodes v as a query string value
func (qv Value) String(v string) {
qv.queryValue.String(v)
}
// Byte encodes v as a query string value
func (qv Value) Byte(v int8) {
qv.queryValue.Byte(v)
}
// Short encodes v as a query string value
func (qv Value) Short(v int16) {
qv.queryValue.Short(v)
}
// Integer encodes v as a query string value
func (qv Value) Integer(v int32) {
qv.queryValue.Integer(v)
}
// Long encodes v as a query string value
func (qv Value) Long(v int64) {
qv.queryValue.Long(v)
}
// Float encodes v as a query string value
func (qv Value) Float(v float32) {
qv.queryValue.Float(v)
}
// Double encodes v as a query string value
func (qv Value) Double(v float64) {
qv.queryValue.Double(v)
}
// BigInteger encodes v as a query string value
func (qv Value) BigInteger(v *big.Int) {
qv.queryValue.BigInteger(v)
}
// BigDecimal encodes v as a query string value
func (qv Value) BigDecimal(v *big.Float) {
qv.queryValue.BigDecimal(v)
}
@@ -1,85 +0,0 @@
package restjson
import (
"encoding/json"
"io"
"strings"
"github.com/aws/smithy-go"
)
// GetErrorInfo util looks for code, __type, and message members in the
// json body. These members are optionally available, and the function
// returns the value of member if it is available. This function is useful to
// identify the error code, msg in a REST JSON error response.
func GetErrorInfo(decoder *json.Decoder) (errorType string, message string, err error) {
var errInfo struct {
Code string
Type string `json:"__type"`
Message string
}
err = decoder.Decode(&errInfo)
if err != nil {
if err == io.EOF {
return errorType, message, nil
}
return errorType, message, err
}
// assign error type
if len(errInfo.Code) != 0 {
errorType = errInfo.Code
} else if len(errInfo.Type) != 0 {
errorType = errInfo.Type
}
// assign error message
if len(errInfo.Message) != 0 {
message = errInfo.Message
}
// sanitize error
if len(errorType) != 0 {
errorType = SanitizeErrorCode(errorType)
}
return errorType, message, nil
}
// SanitizeErrorCode sanitizes the errorCode string .
// The rule for sanitizing is if a `:` character is present, then take only the
// contents before the first : character in the value.
// If a # character is present, then take only the contents after the
// first # character in the value.
func SanitizeErrorCode(errorCode string) string {
if strings.ContainsAny(errorCode, ":") {
errorCode = strings.SplitN(errorCode, ":", 2)[0]
}
if strings.ContainsAny(errorCode, "#") {
errorCode = strings.SplitN(errorCode, "#", 2)[1]
}
return errorCode
}
// GetSmithyGenericAPIError returns smithy generic api error and an error interface.
// Takes in json decoder, and error Code string as args. The function retrieves error message
// and error code from the decoder body. If errorCode of length greater than 0 is passed in as
// an argument, it is used instead.
func GetSmithyGenericAPIError(decoder *json.Decoder, errorCode string) (*smithy.GenericAPIError, error) {
errorType, message, err := GetErrorInfo(decoder)
if err != nil {
return nil, err
}
if len(errorCode) == 0 {
errorCode = errorType
}
return &smithy.GenericAPIError{
Code: errorCode,
Message: message,
}, nil
}
-48
View File
@@ -1,48 +0,0 @@
package xml
import (
"encoding/xml"
"fmt"
"io"
)
// ErrorComponents represents the error response fields
// that will be deserialized from an xml error response body
type ErrorComponents struct {
Code string
Message string
RequestID string
}
// GetErrorResponseComponents returns the error fields from an xml error response body
func GetErrorResponseComponents(r io.Reader, noErrorWrapping bool) (ErrorComponents, error) {
if noErrorWrapping {
var errResponse noWrappedErrorResponse
if err := xml.NewDecoder(r).Decode(&errResponse); err != nil && err != io.EOF {
return ErrorComponents{}, fmt.Errorf("error while deserializing xml error response: %w", err)
}
return ErrorComponents(errResponse), nil
}
var errResponse wrappedErrorResponse
if err := xml.NewDecoder(r).Decode(&errResponse); err != nil && err != io.EOF {
return ErrorComponents{}, fmt.Errorf("error while deserializing xml error response: %w", err)
}
return ErrorComponents(errResponse), nil
}
// noWrappedErrorResponse represents the error response body with
// no internal Error wrapping
type noWrappedErrorResponse struct {
Code string `xml:"Code"`
Message string `xml:"Message"`
RequestID string `xml:"RequestId"`
}
// wrappedErrorResponse represents the error response body
// wrapped within Error
type wrappedErrorResponse struct {
Code string `xml:"Error>Code"`
Message string `xml:"Error>Message"`
RequestID string `xml:"RequestId"`
}
-20
View File
@@ -1,20 +0,0 @@
package ratelimit
import "context"
// None implements a no-op rate limiter which effectively disables client-side
// rate limiting (also known as "retry quotas").
//
// GetToken does nothing and always returns a nil error. The returned
// token-release function does nothing, and always returns a nil error.
//
// AddTokens does nothing and always returns a nil error.
var None = &none{}
type none struct{}
func (*none) GetToken(ctx context.Context, cost uint) (func() error, error) {
return func() error { return nil }, nil
}
func (*none) AddTokens(v uint) error { return nil }
-96
View File
@@ -1,96 +0,0 @@
package ratelimit
import (
"sync"
)
// TokenBucket provides a concurrency safe utility for adding and removing
// tokens from the available token bucket.
type TokenBucket struct {
remainingTokens uint
maxCapacity uint
minCapacity uint
mu sync.Mutex
}
// NewTokenBucket returns an initialized TokenBucket with the capacity
// specified.
func NewTokenBucket(i uint) *TokenBucket {
return &TokenBucket{
remainingTokens: i,
maxCapacity: i,
minCapacity: 1,
}
}
// Retrieve attempts to reduce the available tokens by the amount requested. If
// there are tokens available true will be returned along with the number of
// available tokens remaining. If amount requested is larger than the available
// capacity, false will be returned along with the available capacity. If the
// amount is less than the available capacity, the capacity will be reduced by
// that amount, and the remaining capacity and true will be returned.
func (t *TokenBucket) Retrieve(amount uint) (available uint, retrieved bool) {
t.mu.Lock()
defer t.mu.Unlock()
if amount > t.remainingTokens {
return t.remainingTokens, false
}
t.remainingTokens -= amount
return t.remainingTokens, true
}
// Refund returns the amount of tokens back to the available token bucket, up
// to the initial capacity.
func (t *TokenBucket) Refund(amount uint) {
t.mu.Lock()
defer t.mu.Unlock()
// Capacity cannot exceed max capacity.
t.remainingTokens = uintMin(t.remainingTokens+amount, t.maxCapacity)
}
// Capacity returns the maximum capacity of tokens that the bucket could
// contain.
func (t *TokenBucket) Capacity() uint {
t.mu.Lock()
defer t.mu.Unlock()
return t.maxCapacity
}
// Remaining returns the number of tokens that remaining in the bucket.
func (t *TokenBucket) Remaining() uint {
t.mu.Lock()
defer t.mu.Unlock()
return t.remainingTokens
}
// Resize adjusts the size of the token bucket. Returns the capacity remaining.
func (t *TokenBucket) Resize(size uint) uint {
t.mu.Lock()
defer t.mu.Unlock()
t.maxCapacity = uintMax(size, t.minCapacity)
// Capacity needs to be capped at max capacity, if max size reduced.
t.remainingTokens = uintMin(t.remainingTokens, t.maxCapacity)
return t.remainingTokens
}
func uintMin(a, b uint) uint {
if a < b {
return a
}
return b
}
func uintMax(a, b uint) uint {
if a > b {
return a
}
return b
}
-83
View File
@@ -1,83 +0,0 @@
package ratelimit
import (
"context"
"fmt"
)
type rateToken struct {
tokenCost uint
bucket *TokenBucket
}
func (t rateToken) release() error {
t.bucket.Refund(t.tokenCost)
return nil
}
// TokenRateLimit provides a Token Bucket RateLimiter implementation
// that limits the overall number of retry attempts that can be made across
// operation invocations.
type TokenRateLimit struct {
bucket *TokenBucket
}
// NewTokenRateLimit returns an TokenRateLimit with default values.
// Functional options can configure the retry rate limiter.
func NewTokenRateLimit(tokens uint) *TokenRateLimit {
return &TokenRateLimit{
bucket: NewTokenBucket(tokens),
}
}
type canceledError struct {
Err error
}
func (c canceledError) CanceledError() bool { return true }
func (c canceledError) Unwrap() error { return c.Err }
func (c canceledError) Error() string {
return fmt.Sprintf("canceled, %v", c.Err)
}
// GetToken may cause a available pool of retry quota to be
// decremented. Will return an error if the decremented value can not be
// reduced from the retry quota.
func (l *TokenRateLimit) GetToken(ctx context.Context, cost uint) (func() error, error) {
select {
case <-ctx.Done():
return nil, canceledError{Err: ctx.Err()}
default:
}
if avail, ok := l.bucket.Retrieve(cost); !ok {
return nil, QuotaExceededError{Available: avail, Requested: cost}
}
return rateToken{
tokenCost: cost,
bucket: l.bucket,
}.release, nil
}
// AddTokens increments the token bucket by a fixed amount.
func (l *TokenRateLimit) AddTokens(v uint) error {
l.bucket.Refund(v)
return nil
}
// Remaining returns the number of remaining tokens in the bucket.
func (l *TokenRateLimit) Remaining() uint {
return l.bucket.Remaining()
}
// QuotaExceededError provides the SDK error when the retries for a given
// token bucket have been exhausted.
type QuotaExceededError struct {
Available uint
Requested uint
}
func (e QuotaExceededError) Error() string {
return fmt.Sprintf("retry quota exceeded, %d available, %d requested",
e.Available, e.Requested)
}
-25
View File
@@ -1,25 +0,0 @@
package aws
import (
"fmt"
)
// TODO remove replace with smithy.CanceledError
// RequestCanceledError is the error that will be returned by an API request
// that was canceled. Requests given a Context may return this error when
// canceled.
type RequestCanceledError struct {
Err error
}
// CanceledError returns true to satisfy interfaces checking for canceled errors.
func (*RequestCanceledError) CanceledError() bool { return true }
// Unwrap returns the underlying error, if there was one.
func (e *RequestCanceledError) Unwrap() error {
return e.Err
}
func (e *RequestCanceledError) Error() string {
return fmt.Sprintf("request canceled, %v", e.Err)
}
-156
View File
@@ -1,156 +0,0 @@
package retry
import (
"context"
"fmt"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/internal/sdk"
)
const (
// DefaultRequestCost is the cost of a single request from the adaptive
// rate limited token bucket.
DefaultRequestCost uint = 1
)
// DefaultThrottles provides the set of errors considered throttle errors that
// are checked by default.
var DefaultThrottles = []IsErrorThrottle{
ThrottleErrorCode{
Codes: DefaultThrottleErrorCodes,
},
}
// AdaptiveModeOptions provides the functional options for configuring the
// adaptive retry mode, and delay behavior.
type AdaptiveModeOptions struct {
// If the adaptive token bucket is empty, when an attempt will be made
// AdaptiveMode will sleep until a token is available. This can occur when
// attempts fail with throttle errors. Use this option to disable the sleep
// until token is available, and return error immediately.
FailOnNoAttemptTokens bool
// The cost of an attempt from the AdaptiveMode's adaptive token bucket.
RequestCost uint
// Set of strategies to determine if the attempt failed due to a throttle
// error.
//
// It is safe to append to this list in NewAdaptiveMode's functional options.
Throttles []IsErrorThrottle
// Set of options for standard retry mode that AdaptiveMode is built on top
// of. AdaptiveMode may apply its own defaults to Standard retry mode that
// are different than the defaults of NewStandard. Use these options to
// override the default options.
StandardOptions []func(*StandardOptions)
}
// AdaptiveMode provides an experimental retry strategy that expands on the
// Standard retry strategy, adding client attempt rate limits. The attempt rate
// limit is initially unrestricted, but becomes restricted when the attempt
// fails with for a throttle error. When restricted AdaptiveMode may need to
// sleep before an attempt is made, if too many throttles have been received.
// AdaptiveMode's sleep can be canceled with context cancel. Set
// AdaptiveModeOptions FailOnNoAttemptTokens to change the behavior from sleep,
// to fail fast.
//
// Eventually unrestricted attempt rate limit will be restored once attempts no
// longer are failing due to throttle errors.
type AdaptiveMode struct {
options AdaptiveModeOptions
throttles IsErrorThrottles
retryer aws.RetryerV2
rateLimit *adaptiveRateLimit
}
// NewAdaptiveMode returns an initialized AdaptiveMode retry strategy.
func NewAdaptiveMode(optFns ...func(*AdaptiveModeOptions)) *AdaptiveMode {
o := AdaptiveModeOptions{
RequestCost: DefaultRequestCost,
Throttles: append([]IsErrorThrottle{}, DefaultThrottles...),
}
for _, fn := range optFns {
fn(&o)
}
return &AdaptiveMode{
options: o,
throttles: IsErrorThrottles(o.Throttles),
retryer: NewStandard(o.StandardOptions...),
rateLimit: newAdaptiveRateLimit(),
}
}
// IsErrorRetryable returns if the failed attempt is retryable. This check
// should determine if the error can be retried, or if the error is
// terminal.
func (a *AdaptiveMode) IsErrorRetryable(err error) bool {
return a.retryer.IsErrorRetryable(err)
}
// MaxAttempts returns the maximum number of attempts that can be made for
// an attempt before failing. A value of 0 implies that the attempt should
// be retried until it succeeds if the errors are retryable.
func (a *AdaptiveMode) MaxAttempts() int {
return a.retryer.MaxAttempts()
}
// RetryDelay returns the delay that should be used before retrying the
// attempt. Will return error if the if the delay could not be determined.
func (a *AdaptiveMode) RetryDelay(attempt int, opErr error) (
time.Duration, error,
) {
return a.retryer.RetryDelay(attempt, opErr)
}
// GetRetryToken attempts to deduct the retry cost from the retry token pool.
// Returning the token release function, or error.
func (a *AdaptiveMode) GetRetryToken(ctx context.Context, opErr error) (
releaseToken func(error) error, err error,
) {
return a.retryer.GetRetryToken(ctx, opErr)
}
// GetInitialToken returns the initial attempt token that can increment the
// retry token pool if the attempt is successful.
//
// Deprecated: This method does not provide a way to block using Context,
// nor can it return an error. Use RetryerV2, and GetAttemptToken instead. Only
// present to implement Retryer interface.
func (a *AdaptiveMode) GetInitialToken() (releaseToken func(error) error) {
return nopRelease
}
// GetAttemptToken returns the attempt token that can be used to rate limit
// attempt calls. Will be used by the SDK's retry package's Attempt
// middleware to get an attempt token prior to calling the temp and releasing
// the attempt token after the attempt has been made.
func (a *AdaptiveMode) GetAttemptToken(ctx context.Context) (func(error) error, error) {
for {
acquiredToken, waitTryAgain := a.rateLimit.AcquireToken(a.options.RequestCost)
if acquiredToken {
break
}
if a.options.FailOnNoAttemptTokens {
return nil, fmt.Errorf(
"unable to get attempt token, and FailOnNoAttemptTokens enables")
}
if err := sdk.SleepWithContext(ctx, waitTryAgain); err != nil {
return nil, fmt.Errorf("failed to wait for token to be available, %w", err)
}
}
return a.handleResponse, nil
}
func (a *AdaptiveMode) handleResponse(opErr error) error {
throttled := a.throttles.IsErrorThrottle(opErr).Bool()
a.rateLimit.Update(throttled)
return nil
}
-158
View File
@@ -1,158 +0,0 @@
package retry
import (
"math"
"sync"
"time"
"github.com/aws/aws-sdk-go-v2/internal/sdk"
)
type adaptiveRateLimit struct {
tokenBucketEnabled bool
smooth float64
beta float64
scaleConstant float64
minFillRate float64
fillRate float64
calculatedRate float64
lastRefilled time.Time
measuredTxRate float64
lastTxRateBucket float64
requestCount int64
lastMaxRate float64
lastThrottleTime time.Time
timeWindow float64
tokenBucket *adaptiveTokenBucket
mu sync.Mutex
}
func newAdaptiveRateLimit() *adaptiveRateLimit {
now := sdk.NowTime()
return &adaptiveRateLimit{
smooth: 0.8,
beta: 0.7,
scaleConstant: 0.4,
minFillRate: 0.5,
lastTxRateBucket: math.Floor(timeFloat64Seconds(now)),
lastThrottleTime: now,
tokenBucket: newAdaptiveTokenBucket(0),
}
}
func (a *adaptiveRateLimit) Enable(v bool) {
a.mu.Lock()
defer a.mu.Unlock()
a.tokenBucketEnabled = v
}
func (a *adaptiveRateLimit) AcquireToken(amount uint) (
tokenAcquired bool, waitTryAgain time.Duration,
) {
a.mu.Lock()
defer a.mu.Unlock()
if !a.tokenBucketEnabled {
return true, 0
}
a.tokenBucketRefill()
available, ok := a.tokenBucket.Retrieve(float64(amount))
if !ok {
waitDur := float64Seconds((float64(amount) - available) / a.fillRate)
return false, waitDur
}
return true, 0
}
func (a *adaptiveRateLimit) Update(throttled bool) {
a.mu.Lock()
defer a.mu.Unlock()
a.updateMeasuredRate()
if throttled {
rateToUse := a.measuredTxRate
if a.tokenBucketEnabled {
rateToUse = math.Min(a.measuredTxRate, a.fillRate)
}
a.lastMaxRate = rateToUse
a.calculateTimeWindow()
a.lastThrottleTime = sdk.NowTime()
a.calculatedRate = a.cubicThrottle(rateToUse)
a.tokenBucketEnabled = true
} else {
a.calculateTimeWindow()
a.calculatedRate = a.cubicSuccess(sdk.NowTime())
}
newRate := math.Min(a.calculatedRate, 2*a.measuredTxRate)
a.tokenBucketUpdateRate(newRate)
}
func (a *adaptiveRateLimit) cubicSuccess(t time.Time) float64 {
dt := secondsFloat64(t.Sub(a.lastThrottleTime))
return (a.scaleConstant * math.Pow(dt-a.timeWindow, 3)) + a.lastMaxRate
}
func (a *adaptiveRateLimit) cubicThrottle(rateToUse float64) float64 {
return rateToUse * a.beta
}
func (a *adaptiveRateLimit) calculateTimeWindow() {
a.timeWindow = math.Pow((a.lastMaxRate*(1.-a.beta))/a.scaleConstant, 1./3.)
}
func (a *adaptiveRateLimit) tokenBucketUpdateRate(newRPS float64) {
a.tokenBucketRefill()
a.fillRate = math.Max(newRPS, a.minFillRate)
a.tokenBucket.Resize(newRPS)
}
func (a *adaptiveRateLimit) updateMeasuredRate() {
now := sdk.NowTime()
timeBucket := math.Floor(timeFloat64Seconds(now)*2.) / 2.
a.requestCount++
if timeBucket > a.lastTxRateBucket {
currentRate := float64(a.requestCount) / (timeBucket - a.lastTxRateBucket)
a.measuredTxRate = (currentRate * a.smooth) + (a.measuredTxRate * (1. - a.smooth))
a.requestCount = 0
a.lastTxRateBucket = timeBucket
}
}
func (a *adaptiveRateLimit) tokenBucketRefill() {
now := sdk.NowTime()
if a.lastRefilled.IsZero() {
a.lastRefilled = now
return
}
fillAmount := secondsFloat64(now.Sub(a.lastRefilled)) * a.fillRate
a.tokenBucket.Refund(fillAmount)
a.lastRefilled = now
}
func float64Seconds(v float64) time.Duration {
return time.Duration(v * float64(time.Second))
}
func secondsFloat64(v time.Duration) float64 {
return float64(v) / float64(time.Second)
}
func timeFloat64Seconds(v time.Time) float64 {
return float64(v.UnixNano()) / float64(time.Second)
}
-83
View File
@@ -1,83 +0,0 @@
package retry
import (
"math"
"sync"
)
// adaptiveTokenBucket provides a concurrency safe utility for adding and
// removing tokens from the available token bucket.
type adaptiveTokenBucket struct {
remainingTokens float64
maxCapacity float64
minCapacity float64
mu sync.Mutex
}
// newAdaptiveTokenBucket returns an initialized adaptiveTokenBucket with the
// capacity specified.
func newAdaptiveTokenBucket(i float64) *adaptiveTokenBucket {
return &adaptiveTokenBucket{
remainingTokens: i,
maxCapacity: i,
minCapacity: 1,
}
}
// Retrieve attempts to reduce the available tokens by the amount requested. If
// there are tokens available true will be returned along with the number of
// available tokens remaining. If amount requested is larger than the available
// capacity, false will be returned along with the available capacity. If the
// amount is less than the available capacity, the capacity will be reduced by
// that amount, and the remaining capacity and true will be returned.
func (t *adaptiveTokenBucket) Retrieve(amount float64) (available float64, retrieved bool) {
t.mu.Lock()
defer t.mu.Unlock()
if amount > t.remainingTokens {
return t.remainingTokens, false
}
t.remainingTokens -= amount
return t.remainingTokens, true
}
// Refund returns the amount of tokens back to the available token bucket, up
// to the initial capacity.
func (t *adaptiveTokenBucket) Refund(amount float64) {
t.mu.Lock()
defer t.mu.Unlock()
// Capacity cannot exceed max capacity.
t.remainingTokens = math.Min(t.remainingTokens+amount, t.maxCapacity)
}
// Capacity returns the maximum capacity of tokens that the bucket could
// contain.
func (t *adaptiveTokenBucket) Capacity() float64 {
t.mu.Lock()
defer t.mu.Unlock()
return t.maxCapacity
}
// Remaining returns the number of tokens that remaining in the bucket.
func (t *adaptiveTokenBucket) Remaining() float64 {
t.mu.Lock()
defer t.mu.Unlock()
return t.remainingTokens
}
// Resize adjusts the size of the token bucket. Returns the capacity remaining.
func (t *adaptiveTokenBucket) Resize(size float64) float64 {
t.mu.Lock()
defer t.mu.Unlock()
t.maxCapacity = math.Max(size, t.minCapacity)
// Capacity needs to be capped at max capacity, if max size reduced.
t.remainingTokens = math.Min(t.remainingTokens, t.maxCapacity)
return t.remainingTokens
}
-51
View File
@@ -1,51 +0,0 @@
package retry
import (
"context"
"github.com/aws/smithy-go/metrics"
"github.com/aws/smithy-go/middleware"
)
type attemptMetrics struct {
Attempts metrics.Int64Counter
Errors metrics.Int64Counter
AttemptDuration metrics.Float64Histogram
}
func newAttemptMetrics(meter metrics.Meter) (*attemptMetrics, error) {
m := &attemptMetrics{}
var err error
m.Attempts, err = meter.Int64Counter("client.call.attempts", func(o *metrics.InstrumentOptions) {
o.UnitLabel = "{attempt}"
o.Description = "The number of attempts for an individual operation"
})
if err != nil {
return nil, err
}
m.Errors, err = meter.Int64Counter("client.call.errors", func(o *metrics.InstrumentOptions) {
o.UnitLabel = "{error}"
o.Description = "The number of errors for an operation"
})
if err != nil {
return nil, err
}
m.AttemptDuration, err = meter.Float64Histogram("client.call.attempt_duration", func(o *metrics.InstrumentOptions) {
o.UnitLabel = "s"
o.Description = "The time it takes to connect to the service, send the request, and get back HTTP status code and headers (including time queued waiting to be sent)"
})
if err != nil {
return nil, err
}
return m, nil
}
func withOperationMetadata(ctx context.Context) metrics.RecordMetricOption {
return func(o *metrics.RecordMetricOptions) {
o.Properties.Set("rpc.service", middleware.GetServiceID(ctx))
o.Properties.Set("rpc.method", middleware.GetOperationName(ctx))
}
}
-80
View File
@@ -1,80 +0,0 @@
// Package retry provides interfaces and implementations for SDK request retry behavior.
//
// # Retryer Interface and Implementations
//
// This package defines Retryer interface that is used to either implement custom retry behavior
// or to extend the existing retry implementations provided by the SDK. This package provides a single
// retry implementation: Standard.
//
// # Standard
//
// Standard is the default retryer implementation used by service clients. The standard retryer is a rate limited
// retryer that has a configurable max attempts to limit the number of retry attempts when a retryable error occurs.
// In addition, the retryer uses a configurable token bucket to rate limit the retry attempts across the client,
// and uses an additional delay policy to limit the time between a requests subsequent attempts.
//
// By default the standard retryer uses the DefaultRetryables slice of IsErrorRetryable types to determine whether
// a given error is retryable. By default this list of retryables includes the following:
// - Retrying errors that implement the RetryableError method, and return true.
// - Connection Errors
// - Errors that implement a ConnectionError, Temporary, or Timeout method that return true.
// - Connection Reset Errors.
// - net.OpErr types that are dialing errors or are temporary.
// - HTTP Status Codes: 500, 502, 503, and 504.
// - API Error Codes
// - RequestTimeout, RequestTimeoutException
// - Throttling, ThrottlingException, ThrottledException, RequestThrottledException, TooManyRequestsException,
// RequestThrottled, SlowDown, EC2ThrottledException
// - ProvisionedThroughputExceededException, RequestLimitExceeded, BandwidthLimitExceeded, LimitExceededException
// - TransactionInProgressException, PriorRequestNotComplete
//
// The standard retryer will not retry a request in the event if the context associated with the request
// has been cancelled. Applications must handle this case explicitly if they wish to retry with a different context
// value.
//
// You can configure the standard retryer implementation to fit your applications by constructing a standard retryer
// using the NewStandard function, and providing one more functional argument that mutate the StandardOptions
// structure. StandardOptions provides the ability to modify the token bucket rate limiter, retryable error conditions,
// and the retry delay policy.
//
// For example to modify the default retry attempts for the standard retryer:
//
// // configure the custom retryer
// customRetry := retry.NewStandard(func(o *retry.StandardOptions) {
// o.MaxAttempts = 5
// })
//
// // create a service client with the retryer
// s3.NewFromConfig(cfg, func(o *s3.Options) {
// o.Retryer = customRetry
// })
//
// # Utilities
//
// A number of package functions have been provided to easily wrap retryer implementations in an implementation agnostic
// way. These are:
//
// AddWithErrorCodes - Provides the ability to add additional API error codes that should be considered retryable
// in addition to those considered retryable by the provided retryer.
//
// AddWithMaxAttempts - Provides the ability to set the max number of attempts for retrying a request by wrapping
// a retryer implementation.
//
// AddWithMaxBackoffDelay - Provides the ability to set the max back off delay that can occur before retrying a
// request by wrapping a retryer implementation.
//
// The following package functions have been provided to easily satisfy different retry interfaces to further customize
// a given retryer's behavior:
//
// BackoffDelayerFunc - Can be used to wrap a function to satisfy the BackoffDelayer interface. For example,
// you can use this method to easily create custom back off policies to be used with the
// standard retryer.
//
// IsErrorRetryableFunc - Can be used to wrap a function to satisfy the IsErrorRetryable interface. For example,
// this can be used to extend the standard retryer to add additional logic to determine if an
// error should be retried.
//
// IsErrorTimeoutFunc - Can be used to wrap a function to satisfy IsErrorTimeout interface. For example,
// this can be used to extend the standard retryer to add additional logic to determine if an
// error should be considered a timeout.
package retry
-20
View File
@@ -1,20 +0,0 @@
package retry
import "fmt"
// MaxAttemptsError provides the error when the maximum number of attempts have
// been exceeded.
type MaxAttemptsError struct {
Attempt int
Err error
}
func (e *MaxAttemptsError) Error() string {
return fmt.Sprintf("exceeded maximum number of attempts, %d, %v", e.Attempt, e.Err)
}
// Unwrap returns the nested error causing the max attempts error. Provides the
// implementation for errors.Is and errors.As to unwrap nested errors.
func (e *MaxAttemptsError) Unwrap() error {
return e.Err
}
-49
View File
@@ -1,49 +0,0 @@
package retry
import (
"math"
"time"
"github.com/aws/aws-sdk-go-v2/internal/rand"
"github.com/aws/aws-sdk-go-v2/internal/timeconv"
)
// ExponentialJitterBackoff provides backoff delays with jitter based on the
// number of attempts.
type ExponentialJitterBackoff struct {
maxBackoff time.Duration
// precomputed number of attempts needed to reach max backoff.
maxBackoffAttempts float64
randFloat64 func() (float64, error)
}
// NewExponentialJitterBackoff returns an ExponentialJitterBackoff configured
// for the max backoff.
func NewExponentialJitterBackoff(maxBackoff time.Duration) *ExponentialJitterBackoff {
return &ExponentialJitterBackoff{
maxBackoff: maxBackoff,
maxBackoffAttempts: math.Log2(
float64(maxBackoff) / float64(time.Second)),
randFloat64: rand.CryptoRandFloat64,
}
}
// BackoffDelay returns the duration to wait before the next attempt should be
// made. Returns an error if unable get a duration.
func (j *ExponentialJitterBackoff) BackoffDelay(attempt int, err error) (time.Duration, error) {
if attempt > int(j.maxBackoffAttempts) {
return j.maxBackoff, nil
}
b, err := j.randFloat64()
if err != nil {
return 0, err
}
// [0.0, 1.0) * 2 ^ attempts
ri := int64(1 << uint64(attempt))
delaySeconds := b * float64(ri)
return timeconv.FloatSecondsDur(delaySeconds), nil
}
-52
View File
@@ -1,52 +0,0 @@
package retry
import (
awsmiddle "github.com/aws/aws-sdk-go-v2/aws/middleware"
"github.com/aws/smithy-go/middleware"
)
// attemptResultsKey is a metadata accessor key to retrieve metadata
// for all request attempts.
type attemptResultsKey struct {
}
// GetAttemptResults retrieves attempts results from middleware metadata.
func GetAttemptResults(metadata middleware.Metadata) (AttemptResults, bool) {
m, ok := metadata.Get(attemptResultsKey{}).(AttemptResults)
return m, ok
}
// AttemptResults represents struct containing metadata returned by all request attempts.
type AttemptResults struct {
// Results is a slice consisting attempt result from all request attempts.
// Results are stored in order request attempt is made.
Results []AttemptResult
}
// AttemptResult represents attempt result returned by a single request attempt.
type AttemptResult struct {
// Err is the error if received for the request attempt.
Err error
// Retryable denotes if request may be retried. This states if an
// error is considered retryable.
Retryable bool
// Retried indicates if this request was retried.
Retried bool
// ResponseMetadata is any existing metadata passed via the response middlewares.
ResponseMetadata middleware.Metadata
}
// addAttemptResults adds attempt results to middleware metadata
func addAttemptResults(metadata *middleware.Metadata, v AttemptResults) {
metadata.Set(attemptResultsKey{}, v)
}
// GetRawResponse returns raw response recorded for the attempt result
func (a AttemptResult) GetRawResponse() interface{} {
return awsmiddle.GetRawResponse(a.ResponseMetadata)
}
-418
View File
@@ -1,418 +0,0 @@
package retry
import (
"context"
"errors"
"fmt"
"strconv"
"strings"
"time"
internalcontext "github.com/aws/aws-sdk-go-v2/internal/context"
"github.com/aws/smithy-go"
"github.com/aws/aws-sdk-go-v2/aws"
awsmiddle "github.com/aws/aws-sdk-go-v2/aws/middleware"
"github.com/aws/aws-sdk-go-v2/internal/sdk"
"github.com/aws/smithy-go/logging"
"github.com/aws/smithy-go/metrics"
smithymiddle "github.com/aws/smithy-go/middleware"
"github.com/aws/smithy-go/tracing"
"github.com/aws/smithy-go/transport/http"
)
// RequestCloner is a function that can take an input request type and clone
// the request for use in a subsequent retry attempt.
type RequestCloner func(interface{}) interface{}
type retryMetadata struct {
AttemptNum int
AttemptTime time.Time
MaxAttempts int
AttemptClockSkew time.Duration
}
// Attempt is a Smithy Finalize middleware that handles retry attempts using
// the provided Retryer implementation.
type Attempt struct {
// Enable the logging of retry attempts performed by the SDK. This will
// include logging retry attempts, unretryable errors, and when max
// attempts are reached.
LogAttempts bool
// A Meter instance for recording retry-related metrics.
OperationMeter metrics.Meter
retryer aws.RetryerV2
requestCloner RequestCloner
}
// define the threshold at which we will consider certain kind of errors to be probably
// caused by clock skew
const skewThreshold = 4 * time.Minute
// NewAttemptMiddleware returns a new Attempt retry middleware.
func NewAttemptMiddleware(retryer aws.Retryer, requestCloner RequestCloner, optFns ...func(*Attempt)) *Attempt {
m := &Attempt{
retryer: wrapAsRetryerV2(retryer),
requestCloner: requestCloner,
}
for _, fn := range optFns {
fn(m)
}
if m.OperationMeter == nil {
m.OperationMeter = metrics.NopMeterProvider{}.Meter("")
}
return m
}
// ID returns the middleware identifier
func (r *Attempt) ID() string { return "Retry" }
func (r Attempt) logf(logger logging.Logger, classification logging.Classification, format string, v ...interface{}) {
if !r.LogAttempts {
return
}
logger.Logf(classification, format, v...)
}
// HandleFinalize utilizes the provider Retryer implementation to attempt
// retries over the next handler
func (r *Attempt) HandleFinalize(ctx context.Context, in smithymiddle.FinalizeInput, next smithymiddle.FinalizeHandler) (
out smithymiddle.FinalizeOutput, metadata smithymiddle.Metadata, err error,
) {
var attemptNum int
var attemptClockSkew time.Duration
var attemptResults AttemptResults
maxAttempts := r.retryer.MaxAttempts()
releaseRetryToken := nopRelease
retryMetrics, err := newAttemptMetrics(r.OperationMeter)
if err != nil {
return out, metadata, err
}
for {
attemptNum++
attemptInput := in
attemptInput.Request = r.requestCloner(attemptInput.Request)
// Record the metadata for the for attempt being started.
attemptCtx := setRetryMetadata(ctx, retryMetadata{
AttemptNum: attemptNum,
AttemptTime: sdk.NowTime().UTC(),
MaxAttempts: maxAttempts,
AttemptClockSkew: attemptClockSkew,
})
// Setting clock skew to be used on other context (like signing)
ctx = internalcontext.SetAttemptSkewContext(ctx, attemptClockSkew)
var attemptResult AttemptResult
attemptCtx, span := tracing.StartSpan(attemptCtx, "Attempt", func(o *tracing.SpanOptions) {
o.Properties.Set("operation.attempt", attemptNum)
})
retryMetrics.Attempts.Add(ctx, 1, withOperationMetadata(ctx))
start := sdk.NowTime()
out, attemptResult, releaseRetryToken, err = r.handleAttempt(attemptCtx, attemptInput, releaseRetryToken, next)
elapsed := sdk.NowTime().Sub(start)
retryMetrics.AttemptDuration.Record(ctx, float64(elapsed)/1e9, withOperationMetadata(ctx))
if err != nil {
retryMetrics.Errors.Add(ctx, 1, withOperationMetadata(ctx), func(o *metrics.RecordMetricOptions) {
o.Properties.Set("exception.type", errorType(err))
})
}
span.End()
attemptClockSkew, _ = awsmiddle.GetAttemptSkew(attemptResult.ResponseMetadata)
// AttemptResult Retried states that the attempt was not successful, and
// should be retried.
shouldRetry := attemptResult.Retried
// Add attempt metadata to list of all attempt metadata
attemptResults.Results = append(attemptResults.Results, attemptResult)
if !shouldRetry {
// Ensure the last response's metadata is used as the bases for result
// metadata returned by the stack. The Slice of attempt results
// will be added to this cloned metadata.
metadata = attemptResult.ResponseMetadata.Clone()
break
}
}
addAttemptResults(&metadata, attemptResults)
return out, metadata, err
}
// handleAttempt handles an individual request attempt.
func (r *Attempt) handleAttempt(
ctx context.Context, in smithymiddle.FinalizeInput, releaseRetryToken func(error) error, next smithymiddle.FinalizeHandler,
) (
out smithymiddle.FinalizeOutput, attemptResult AttemptResult, _ func(error) error, err error,
) {
defer func() {
attemptResult.Err = err
}()
// Short circuit if this attempt never can succeed because the context is
// canceled. This reduces the chance of token pools being modified for
// attempts that will not be made
select {
case <-ctx.Done():
return out, attemptResult, nopRelease, ctx.Err()
default:
}
//------------------------------
// Get Attempt Token
//------------------------------
releaseAttemptToken, err := r.retryer.GetAttemptToken(ctx)
if err != nil {
return out, attemptResult, nopRelease, fmt.Errorf(
"failed to get retry Send token, %w", err)
}
//------------------------------
// Send Attempt
//------------------------------
logger := smithymiddle.GetLogger(ctx)
service, operation := awsmiddle.GetServiceID(ctx), awsmiddle.GetOperationName(ctx)
retryMetadata, _ := getRetryMetadata(ctx)
attemptNum := retryMetadata.AttemptNum
maxAttempts := retryMetadata.MaxAttempts
// Following attempts must ensure the request payload stream starts in a
// rewound state.
if attemptNum > 1 {
if rewindable, ok := in.Request.(interface{ RewindStream() error }); ok {
if rewindErr := rewindable.RewindStream(); rewindErr != nil {
return out, attemptResult, nopRelease, fmt.Errorf(
"failed to rewind transport stream for retry, %w", rewindErr)
}
}
r.logf(logger, logging.Debug, "retrying request %s/%s, attempt %d",
service, operation, attemptNum)
}
var metadata smithymiddle.Metadata
out, metadata, err = next.HandleFinalize(ctx, in)
attemptResult.ResponseMetadata = metadata
//------------------------------
// Bookkeeping
//------------------------------
// Release the retry token based on the state of the attempt's error (if any).
if releaseError := releaseRetryToken(err); releaseError != nil && err != nil {
return out, attemptResult, nopRelease, fmt.Errorf(
"failed to release retry token after request error, %w", err)
}
// Release the attempt token based on the state of the attempt's error (if any).
if releaseError := releaseAttemptToken(err); releaseError != nil && err != nil {
return out, attemptResult, nopRelease, fmt.Errorf(
"failed to release initial token after request error, %w", err)
}
// If there was no error making the attempt, nothing further to do. There
// will be nothing to retry.
if err == nil {
return out, attemptResult, nopRelease, err
}
err = wrapAsClockSkew(ctx, err)
//------------------------------
// Is Retryable and Should Retry
//------------------------------
// If the attempt failed with an unretryable error, nothing further to do
// but return, and inform the caller about the terminal failure.
retryable := r.retryer.IsErrorRetryable(err)
if !retryable {
r.logf(logger, logging.Debug, "request failed with unretryable error %v", err)
return out, attemptResult, nopRelease, err
}
// set retryable to true
attemptResult.Retryable = true
// Once the maximum number of attempts have been exhausted there is nothing
// further to do other than inform the caller about the terminal failure.
if maxAttempts > 0 && attemptNum >= maxAttempts {
r.logf(logger, logging.Debug, "max retry attempts exhausted, max %d", maxAttempts)
err = &MaxAttemptsError{
Attempt: attemptNum,
Err: err,
}
return out, attemptResult, nopRelease, err
}
//------------------------------
// Get Retry (aka Retry Quota) Token
//------------------------------
// Get a retry token that will be released after the
releaseRetryToken, retryTokenErr := r.retryer.GetRetryToken(ctx, err)
if retryTokenErr != nil {
return out, attemptResult, nopRelease, errors.Join(err, retryTokenErr)
}
//------------------------------
// Retry Delay and Sleep
//------------------------------
// Get the retry delay before another attempt can be made, and sleep for
// that time. Potentially early exist if the sleep is canceled via the
// context.
retryDelay, reqErr := r.retryer.RetryDelay(attemptNum, err)
if reqErr != nil {
return out, attemptResult, releaseRetryToken, reqErr
}
if reqErr = sdk.SleepWithContext(ctx, retryDelay); reqErr != nil {
err = &aws.RequestCanceledError{Err: reqErr}
return out, attemptResult, releaseRetryToken, err
}
// The request should be re-attempted.
attemptResult.Retried = true
return out, attemptResult, releaseRetryToken, err
}
// errors that, if detected when we know there's a clock skew,
// can be retried and have a high chance of success
var possibleSkewCodes = map[string]struct{}{
"InvalidSignatureException": {},
"SignatureDoesNotMatch": {},
"AuthFailure": {},
}
var definiteSkewCodes = map[string]struct{}{
"RequestExpired": {},
"RequestInTheFuture": {},
"RequestTimeTooSkewed": {},
}
// wrapAsClockSkew checks if this error could be related to a clock skew
// error and if so, wrap the error.
func wrapAsClockSkew(ctx context.Context, err error) error {
var v interface{ ErrorCode() string }
if !errors.As(err, &v) {
return err
}
if _, ok := definiteSkewCodes[v.ErrorCode()]; ok {
return &retryableClockSkewError{Err: err}
}
_, isPossibleSkewCode := possibleSkewCodes[v.ErrorCode()]
if skew := internalcontext.GetAttemptSkewContext(ctx); skew > skewThreshold && isPossibleSkewCode {
return &retryableClockSkewError{Err: err}
}
return err
}
// MetricsHeader attaches SDK request metric header for retries to the transport
type MetricsHeader struct{}
// ID returns the middleware identifier
func (r *MetricsHeader) ID() string {
return "RetryMetricsHeader"
}
// HandleFinalize attaches the SDK request metric header to the transport layer
func (r MetricsHeader) HandleFinalize(ctx context.Context, in smithymiddle.FinalizeInput, next smithymiddle.FinalizeHandler) (
out smithymiddle.FinalizeOutput, metadata smithymiddle.Metadata, err error,
) {
retryMetadata, _ := getRetryMetadata(ctx)
const retryMetricHeader = "Amz-Sdk-Request"
var parts []string
parts = append(parts, "attempt="+strconv.Itoa(retryMetadata.AttemptNum))
if retryMetadata.MaxAttempts != 0 {
parts = append(parts, "max="+strconv.Itoa(retryMetadata.MaxAttempts))
}
var ttl time.Time
if deadline, ok := ctx.Deadline(); ok {
ttl = deadline
}
// Only append the TTL if it can be determined.
if !ttl.IsZero() && retryMetadata.AttemptClockSkew > 0 {
const unixTimeFormat = "20060102T150405Z"
ttl = ttl.Add(retryMetadata.AttemptClockSkew)
parts = append(parts, "ttl="+ttl.Format(unixTimeFormat))
}
switch req := in.Request.(type) {
case *http.Request:
req.Header[retryMetricHeader] = append(req.Header[retryMetricHeader][:0], strings.Join(parts, "; "))
default:
return out, metadata, fmt.Errorf("unknown transport type %T", req)
}
return next.HandleFinalize(ctx, in)
}
type retryMetadataKey struct{}
// getRetryMetadata retrieves retryMetadata from the context and a bool
// indicating if it was set.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func getRetryMetadata(ctx context.Context) (metadata retryMetadata, ok bool) {
metadata, ok = smithymiddle.GetStackValue(ctx, retryMetadataKey{}).(retryMetadata)
return metadata, ok
}
// setRetryMetadata sets the retryMetadata on the context.
//
// Scoped to stack values. Use github.com/aws/smithy-go/middleware#ClearStackValues
// to clear all stack values.
func setRetryMetadata(ctx context.Context, metadata retryMetadata) context.Context {
return smithymiddle.WithStackValue(ctx, retryMetadataKey{}, metadata)
}
// AddRetryMiddlewaresOptions is the set of options that can be passed to
// AddRetryMiddlewares for configuring retry associated middleware.
type AddRetryMiddlewaresOptions struct {
Retryer aws.Retryer
// Enable the logging of retry attempts performed by the SDK. This will
// include logging retry attempts, unretryable errors, and when max
// attempts are reached.
LogRetryAttempts bool
}
// AddRetryMiddlewares adds retry middleware to operation middleware stack
func AddRetryMiddlewares(stack *smithymiddle.Stack, options AddRetryMiddlewaresOptions) error {
attempt := NewAttemptMiddleware(options.Retryer, http.RequestCloner, func(middleware *Attempt) {
middleware.LogAttempts = options.LogRetryAttempts
})
// index retry to before signing, if signing exists
if err := stack.Finalize.Insert(attempt, "Signing", smithymiddle.Before); err != nil {
return err
}
if err := stack.Finalize.Insert(&MetricsHeader{}, attempt.ID(), smithymiddle.After); err != nil {
return err
}
return nil
}
// Determines the value of exception.type for metrics purposes. We prefer an
// API-specific error code, otherwise it's just the Go type for the value.
func errorType(err error) string {
var terr smithy.APIError
if errors.As(err, &terr) {
return terr.ErrorCode()
}
return fmt.Sprintf("%T", err)
}
-90
View File
@@ -1,90 +0,0 @@
package retry
import (
"context"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
)
// AddWithErrorCodes returns a Retryer with additional error codes considered
// for determining if the error should be retried.
func AddWithErrorCodes(r aws.Retryer, codes ...string) aws.Retryer {
retryable := &RetryableErrorCode{
Codes: map[string]struct{}{},
}
for _, c := range codes {
retryable.Codes[c] = struct{}{}
}
return &withIsErrorRetryable{
RetryerV2: wrapAsRetryerV2(r),
Retryable: retryable,
}
}
type withIsErrorRetryable struct {
aws.RetryerV2
Retryable IsErrorRetryable
}
func (r *withIsErrorRetryable) IsErrorRetryable(err error) bool {
if v := r.Retryable.IsErrorRetryable(err); v != aws.UnknownTernary {
return v.Bool()
}
return r.RetryerV2.IsErrorRetryable(err)
}
// AddWithMaxAttempts returns a Retryer with MaxAttempts set to the value
// specified.
func AddWithMaxAttempts(r aws.Retryer, max int) aws.Retryer {
return &withMaxAttempts{
RetryerV2: wrapAsRetryerV2(r),
Max: max,
}
}
type withMaxAttempts struct {
aws.RetryerV2
Max int
}
func (w *withMaxAttempts) MaxAttempts() int {
return w.Max
}
// AddWithMaxBackoffDelay returns a retryer wrapping the passed in retryer
// overriding the RetryDelay behavior for a alternate minimum initial backoff
// delay.
func AddWithMaxBackoffDelay(r aws.Retryer, delay time.Duration) aws.Retryer {
return &withMaxBackoffDelay{
RetryerV2: wrapAsRetryerV2(r),
backoff: NewExponentialJitterBackoff(delay),
}
}
type withMaxBackoffDelay struct {
aws.RetryerV2
backoff *ExponentialJitterBackoff
}
func (r *withMaxBackoffDelay) RetryDelay(attempt int, err error) (time.Duration, error) {
return r.backoff.BackoffDelay(attempt, err)
}
type wrappedAsRetryerV2 struct {
aws.Retryer
}
func wrapAsRetryerV2(r aws.Retryer) aws.RetryerV2 {
v, ok := r.(aws.RetryerV2)
if !ok {
v = wrappedAsRetryerV2{Retryer: r}
}
return v
}
func (w wrappedAsRetryerV2) GetAttemptToken(context.Context) (func(error) error, error) {
return w.Retryer.GetInitialToken(), nil
}
-228
View File
@@ -1,228 +0,0 @@
package retry
import (
"errors"
"fmt"
"net"
"net/url"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
)
// IsErrorRetryable provides the interface of an implementation to determine if
// a error as the result of an operation is retryable.
type IsErrorRetryable interface {
IsErrorRetryable(error) aws.Ternary
}
// IsErrorRetryables is a collection of checks to determine of the error is
// retryable. Iterates through the checks and returns the state of retryable
// if any check returns something other than unknown.
type IsErrorRetryables []IsErrorRetryable
// IsErrorRetryable returns if the error is retryable if any of the checks in
// the list return a value other than unknown.
func (r IsErrorRetryables) IsErrorRetryable(err error) aws.Ternary {
for _, re := range r {
if v := re.IsErrorRetryable(err); v != aws.UnknownTernary {
return v
}
}
return aws.UnknownTernary
}
// IsErrorRetryableFunc wraps a function with the IsErrorRetryable interface.
type IsErrorRetryableFunc func(error) aws.Ternary
// IsErrorRetryable returns if the error is retryable.
func (fn IsErrorRetryableFunc) IsErrorRetryable(err error) aws.Ternary {
return fn(err)
}
// RetryableError is an IsErrorRetryable implementation which uses the
// optional interface Retryable on the error value to determine if the error is
// retryable.
type RetryableError struct{}
// IsErrorRetryable returns if the error is retryable if it satisfies the
// Retryable interface, and returns if the attempt should be retried.
func (RetryableError) IsErrorRetryable(err error) aws.Ternary {
var v interface{ RetryableError() bool }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
return aws.BoolTernary(v.RetryableError())
}
// NoRetryCanceledError detects if the error was an request canceled error and
// returns if so.
type NoRetryCanceledError struct{}
// IsErrorRetryable returns the error is not retryable if the request was
// canceled.
func (NoRetryCanceledError) IsErrorRetryable(err error) aws.Ternary {
var v interface{ CanceledError() bool }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
if v.CanceledError() {
return aws.FalseTernary
}
return aws.UnknownTernary
}
// RetryableConnectionError determines if the underlying error is an HTTP
// connection and returns if it should be retried.
//
// Includes errors such as connection reset, connection refused, net dial,
// temporary, and timeout errors.
type RetryableConnectionError struct{}
// IsErrorRetryable returns if the error is caused by and HTTP connection
// error, and should be retried.
func (r RetryableConnectionError) IsErrorRetryable(err error) aws.Ternary {
if err == nil {
return aws.UnknownTernary
}
var retryable bool
var conErr interface{ ConnectionError() bool }
var tempErr interface{ Temporary() bool }
var timeoutErr interface{ Timeout() bool }
var urlErr *url.Error
var netOpErr *net.OpError
var dnsError *net.DNSError
if errors.As(err, &dnsError) {
// NXDOMAIN errors should not be retried
if dnsError.IsNotFound {
return aws.BoolTernary(false)
}
// if !dnsError.Temporary(), error may or may not be temporary,
// (i.e. !Temporary() =/=> !retryable) so we should fall through to
// remaining checks
if dnsError.Temporary() {
return aws.BoolTernary(true)
}
}
switch {
case errors.As(err, &conErr) && conErr.ConnectionError():
retryable = true
case strings.Contains(err.Error(), "use of closed network connection"):
fallthrough
case strings.Contains(err.Error(), "connection reset"):
// The errors "connection reset" and "use of closed network connection"
// are effectively the same. It appears to be the difference between
// sync and async read of TCP RST in the stdlib's net.Conn read loop.
// see #2737
retryable = true
case errors.As(err, &urlErr):
// Refused connections should be retried as the service may not yet be
// running on the port. Go TCP dial considers refused connections as
// not temporary.
if strings.Contains(urlErr.Error(), "connection refused") {
retryable = true
} else {
return r.IsErrorRetryable(errors.Unwrap(urlErr))
}
case errors.As(err, &netOpErr):
// Network dial, or temporary network errors are always retryable.
if strings.EqualFold(netOpErr.Op, "dial") || netOpErr.Temporary() {
retryable = true
} else {
return r.IsErrorRetryable(errors.Unwrap(netOpErr))
}
case errors.As(err, &tempErr) && tempErr.Temporary():
// Fallback to the generic temporary check, with temporary errors
// retryable.
retryable = true
case errors.As(err, &timeoutErr) && timeoutErr.Timeout():
// Fallback to the generic timeout check, with timeout errors
// retryable.
retryable = true
default:
return aws.UnknownTernary
}
return aws.BoolTernary(retryable)
}
// RetryableHTTPStatusCode provides a IsErrorRetryable based on HTTP status
// codes.
type RetryableHTTPStatusCode struct {
Codes map[int]struct{}
}
// IsErrorRetryable return if the passed in error is retryable based on the
// HTTP status code.
func (r RetryableHTTPStatusCode) IsErrorRetryable(err error) aws.Ternary {
var v interface{ HTTPStatusCode() int }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
_, ok := r.Codes[v.HTTPStatusCode()]
if !ok {
return aws.UnknownTernary
}
return aws.TrueTernary
}
// RetryableErrorCode determines if an attempt should be retried based on the
// API error code.
type RetryableErrorCode struct {
Codes map[string]struct{}
}
// IsErrorRetryable return if the error is retryable based on the error codes.
// Returns unknown if the error doesn't have a code or it is unknown.
func (r RetryableErrorCode) IsErrorRetryable(err error) aws.Ternary {
var v interface{ ErrorCode() string }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
_, ok := r.Codes[v.ErrorCode()]
if !ok {
return aws.UnknownTernary
}
return aws.TrueTernary
}
// retryableClockSkewError marks errors that can be caused by clock skew
// (difference between server time and client time).
// This is returned when there's certain confidence that adjusting the client time
// could allow a retry to succeed
type retryableClockSkewError struct{ Err error }
func (e *retryableClockSkewError) Error() string {
return fmt.Sprintf("Probable clock skew error: %v", e.Err)
}
// Unwrap returns the wrapped error.
func (e *retryableClockSkewError) Unwrap() error {
return e.Err
}
// RetryableError allows the retryer to retry this request
func (e *retryableClockSkewError) RetryableError() bool {
return true
}
-269
View File
@@ -1,269 +0,0 @@
package retry
import (
"context"
"fmt"
"time"
"github.com/aws/aws-sdk-go-v2/aws/ratelimit"
)
// BackoffDelayer provides the interface for determining the delay to before
// another request attempt, that previously failed.
type BackoffDelayer interface {
BackoffDelay(attempt int, err error) (time.Duration, error)
}
// BackoffDelayerFunc provides a wrapper around a function to determine the
// backoff delay of an attempt retry.
type BackoffDelayerFunc func(int, error) (time.Duration, error)
// BackoffDelay returns the delay before attempt to retry a request.
func (fn BackoffDelayerFunc) BackoffDelay(attempt int, err error) (time.Duration, error) {
return fn(attempt, err)
}
const (
// DefaultMaxAttempts is the maximum of attempts for an API request
DefaultMaxAttempts int = 3
// DefaultMaxBackoff is the maximum back off delay between attempts
DefaultMaxBackoff time.Duration = 20 * time.Second
)
// Default retry token quota values.
const (
DefaultRetryRateTokens uint = 500
DefaultRetryCost uint = 5
DefaultRetryTimeoutCost uint = 10
DefaultNoRetryIncrement uint = 1
)
// DefaultRetryableHTTPStatusCodes is the default set of HTTP status codes the SDK
// should consider as retryable errors.
var DefaultRetryableHTTPStatusCodes = map[int]struct{}{
500: {},
502: {},
503: {},
504: {},
}
// DefaultRetryableErrorCodes provides the set of API error codes that should
// be retried.
var DefaultRetryableErrorCodes = map[string]struct{}{
"RequestTimeout": {},
"RequestTimeoutException": {},
}
// DefaultThrottleErrorCodes provides the set of API error codes that are
// considered throttle errors.
var DefaultThrottleErrorCodes = map[string]struct{}{
"Throttling": {},
"ThrottlingException": {},
"ThrottledException": {},
"RequestThrottledException": {},
"TooManyRequestsException": {},
"ProvisionedThroughputExceededException": {},
"TransactionInProgressException": {},
"RequestLimitExceeded": {},
"BandwidthLimitExceeded": {},
"LimitExceededException": {},
"RequestThrottled": {},
"SlowDown": {},
"PriorRequestNotComplete": {},
"EC2ThrottledException": {},
}
// DefaultRetryables provides the set of retryable checks that are used by
// default.
var DefaultRetryables = []IsErrorRetryable{
NoRetryCanceledError{},
RetryableError{},
RetryableConnectionError{},
RetryableHTTPStatusCode{
Codes: DefaultRetryableHTTPStatusCodes,
},
RetryableErrorCode{
Codes: DefaultRetryableErrorCodes,
},
RetryableErrorCode{
Codes: DefaultThrottleErrorCodes,
},
}
// DefaultTimeouts provides the set of timeout checks that are used by default.
var DefaultTimeouts = []IsErrorTimeout{
TimeouterError{},
}
// StandardOptions provides the functional options for configuring the standard
// retryable, and delay behavior.
type StandardOptions struct {
// Maximum number of attempts that should be made.
MaxAttempts int
// MaxBackoff duration between retried attempts.
MaxBackoff time.Duration
// Provides the backoff strategy the retryer will use to determine the
// delay between retry attempts.
Backoff BackoffDelayer
// Set of strategies to determine if the attempt should be retried based on
// the error response received.
//
// It is safe to append to this list in NewStandard's functional options.
Retryables []IsErrorRetryable
// Set of strategies to determine if the attempt failed due to a timeout
// error.
//
// It is safe to append to this list in NewStandard's functional options.
Timeouts []IsErrorTimeout
// Provides the rate limiting strategy for rate limiting attempt retries
// across all attempts the retryer is being used with.
//
// A RateLimiter operates as a token bucket with a set capacity, where
// attempt failures events consume tokens. A retry attempt that attempts to
// consume more tokens than what's available results in operation failure.
// The default implementation is parameterized as follows:
// - a capacity of 500 (DefaultRetryRateTokens)
// - a retry caused by a timeout costs 10 tokens (DefaultRetryCost)
// - a retry caused by other errors costs 5 tokens (DefaultRetryTimeoutCost)
// - an operation that succeeds on the 1st attempt adds 1 token (DefaultNoRetryIncrement)
//
// You can disable rate limiting by setting this field to ratelimit.None.
RateLimiter RateLimiter
// The cost to deduct from the RateLimiter's token bucket per retry.
RetryCost uint
// The cost to deduct from the RateLimiter's token bucket per retry caused
// by timeout error.
RetryTimeoutCost uint
// The cost to payback to the RateLimiter's token bucket for successful
// attempts.
NoRetryIncrement uint
}
// RateLimiter provides the interface for limiting the rate of attempt retries
// allowed by the retryer.
type RateLimiter interface {
GetToken(ctx context.Context, cost uint) (releaseToken func() error, err error)
AddTokens(uint) error
}
// Standard is the standard retry pattern for the SDK. It uses a set of
// retryable checks to determine of the failed attempt should be retried, and
// what retry delay should be used.
type Standard struct {
options StandardOptions
timeout IsErrorTimeout
retryable IsErrorRetryable
backoff BackoffDelayer
}
// NewStandard initializes a standard retry behavior with defaults that can be
// overridden via functional options.
func NewStandard(fnOpts ...func(*StandardOptions)) *Standard {
o := StandardOptions{
MaxAttempts: DefaultMaxAttempts,
MaxBackoff: DefaultMaxBackoff,
Retryables: append([]IsErrorRetryable{}, DefaultRetryables...),
Timeouts: append([]IsErrorTimeout{}, DefaultTimeouts...),
RateLimiter: ratelimit.NewTokenRateLimit(DefaultRetryRateTokens),
RetryCost: DefaultRetryCost,
RetryTimeoutCost: DefaultRetryTimeoutCost,
NoRetryIncrement: DefaultNoRetryIncrement,
}
for _, fn := range fnOpts {
fn(&o)
}
if o.MaxAttempts <= 0 {
o.MaxAttempts = DefaultMaxAttempts
}
backoff := o.Backoff
if backoff == nil {
backoff = NewExponentialJitterBackoff(o.MaxBackoff)
}
return &Standard{
options: o,
backoff: backoff,
retryable: IsErrorRetryables(o.Retryables),
timeout: IsErrorTimeouts(o.Timeouts),
}
}
// MaxAttempts returns the maximum number of attempts that can be made for a
// request before failing.
func (s *Standard) MaxAttempts() int {
return s.options.MaxAttempts
}
// IsErrorRetryable returns if the error is can be retried or not. Should not
// consider the number of attempts made.
func (s *Standard) IsErrorRetryable(err error) bool {
return s.retryable.IsErrorRetryable(err).Bool()
}
// RetryDelay returns the delay to use before another request attempt is made.
func (s *Standard) RetryDelay(attempt int, err error) (time.Duration, error) {
return s.backoff.BackoffDelay(attempt, err)
}
// GetAttemptToken returns the token to be released after then attempt completes.
// The release token will add NoRetryIncrement to the RateLimiter token pool if
// the attempt was successful. If the attempt failed, nothing will be done.
func (s *Standard) GetAttemptToken(context.Context) (func(error) error, error) {
return s.GetInitialToken(), nil
}
// GetInitialToken returns a token for adding the NoRetryIncrement to the
// RateLimiter token if the attempt completed successfully without error.
//
// InitialToken applies to result of the each attempt, including the first.
// Whereas the RetryToken applies to the result of subsequent attempts.
//
// Deprecated: use GetAttemptToken instead.
func (s *Standard) GetInitialToken() func(error) error {
return releaseToken(s.noRetryIncrement).release
}
func (s *Standard) noRetryIncrement() error {
return s.options.RateLimiter.AddTokens(s.options.NoRetryIncrement)
}
// GetRetryToken attempts to deduct the retry cost from the retry token pool.
// Returning the token release function, or error.
func (s *Standard) GetRetryToken(ctx context.Context, opErr error) (func(error) error, error) {
cost := s.options.RetryCost
if s.timeout.IsErrorTimeout(opErr).Bool() {
cost = s.options.RetryTimeoutCost
}
fn, err := s.options.RateLimiter.GetToken(ctx, cost)
if err != nil {
return nil, fmt.Errorf("failed to get rate limit token, %w", err)
}
return releaseToken(fn).release, nil
}
func nopRelease(error) error { return nil }
type releaseToken func() error
func (f releaseToken) release(err error) error {
if err != nil {
return nil
}
return f()
}
-60
View File
@@ -1,60 +0,0 @@
package retry
import (
"errors"
"github.com/aws/aws-sdk-go-v2/aws"
)
// IsErrorThrottle provides the interface of an implementation to determine if
// a error response from an operation is a throttling error.
type IsErrorThrottle interface {
IsErrorThrottle(error) aws.Ternary
}
// IsErrorThrottles is a collection of checks to determine of the error a
// throttle error. Iterates through the checks and returns the state of
// throttle if any check returns something other than unknown.
type IsErrorThrottles []IsErrorThrottle
// IsErrorThrottle returns if the error is a throttle error if any of the
// checks in the list return a value other than unknown.
func (r IsErrorThrottles) IsErrorThrottle(err error) aws.Ternary {
for _, re := range r {
if v := re.IsErrorThrottle(err); v != aws.UnknownTernary {
return v
}
}
return aws.UnknownTernary
}
// IsErrorThrottleFunc wraps a function with the IsErrorThrottle interface.
type IsErrorThrottleFunc func(error) aws.Ternary
// IsErrorThrottle returns if the error is a throttle error.
func (fn IsErrorThrottleFunc) IsErrorThrottle(err error) aws.Ternary {
return fn(err)
}
// ThrottleErrorCode determines if an attempt should be retried based on the
// API error code.
type ThrottleErrorCode struct {
Codes map[string]struct{}
}
// IsErrorThrottle return if the error is a throttle error based on the error
// codes. Returns unknown if the error doesn't have a code or it is unknown.
func (r ThrottleErrorCode) IsErrorThrottle(err error) aws.Ternary {
var v interface{ ErrorCode() string }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
_, ok := r.Codes[v.ErrorCode()]
if !ok {
return aws.UnknownTernary
}
return aws.TrueTernary
}
-52
View File
@@ -1,52 +0,0 @@
package retry
import (
"errors"
"github.com/aws/aws-sdk-go-v2/aws"
)
// IsErrorTimeout provides the interface of an implementation to determine if
// a error matches.
type IsErrorTimeout interface {
IsErrorTimeout(err error) aws.Ternary
}
// IsErrorTimeouts is a collection of checks to determine of the error is
// retryable. Iterates through the checks and returns the state of retryable
// if any check returns something other than unknown.
type IsErrorTimeouts []IsErrorTimeout
// IsErrorTimeout returns if the error is retryable if any of the checks in
// the list return a value other than unknown.
func (ts IsErrorTimeouts) IsErrorTimeout(err error) aws.Ternary {
for _, t := range ts {
if v := t.IsErrorTimeout(err); v != aws.UnknownTernary {
return v
}
}
return aws.UnknownTernary
}
// IsErrorTimeoutFunc wraps a function with the IsErrorTimeout interface.
type IsErrorTimeoutFunc func(error) aws.Ternary
// IsErrorTimeout returns if the error is retryable.
func (fn IsErrorTimeoutFunc) IsErrorTimeout(err error) aws.Ternary {
return fn(err)
}
// TimeouterError provides the IsErrorTimeout implementation for determining if
// an error is a timeout based on type with the Timeout method.
type TimeouterError struct{}
// IsErrorTimeout returns if the error is a timeout error.
func (t TimeouterError) IsErrorTimeout(err error) aws.Ternary {
var v interface{ Timeout() bool }
if !errors.As(err, &v) {
return aws.UnknownTernary
}
return aws.BoolTernary(v.Timeout())
}
-127
View File
@@ -1,127 +0,0 @@
package aws
import (
"context"
"fmt"
"time"
)
// RetryMode provides the mode the API client will use to create a retryer
// based on.
type RetryMode string
const (
// RetryModeStandard model provides rate limited retry attempts with
// exponential backoff delay.
RetryModeStandard RetryMode = "standard"
// RetryModeAdaptive model provides attempt send rate limiting on throttle
// responses in addition to standard mode's retry rate limiting.
//
// Adaptive retry mode is experimental and is subject to change in the
// future.
RetryModeAdaptive RetryMode = "adaptive"
)
// ParseRetryMode attempts to parse a RetryMode from the given string.
// Returning error if the value is not a known RetryMode.
func ParseRetryMode(v string) (mode RetryMode, err error) {
switch v {
case "standard":
return RetryModeStandard, nil
case "adaptive":
return RetryModeAdaptive, nil
default:
return mode, fmt.Errorf("unknown RetryMode, %v", v)
}
}
func (m RetryMode) String() string { return string(m) }
// Retryer is an interface to determine if a given error from a
// attempt should be retried, and if so what backoff delay to apply. The
// default implementation used by most services is the retry package's Standard
// type. Which contains basic retry logic using exponential backoff.
type Retryer interface {
// IsErrorRetryable returns if the failed attempt is retryable. This check
// should determine if the error can be retried, or if the error is
// terminal.
IsErrorRetryable(error) bool
// MaxAttempts returns the maximum number of attempts that can be made for
// an attempt before failing. A value of 0 implies that the attempt should
// be retried until it succeeds if the errors are retryable.
MaxAttempts() int
// RetryDelay returns the delay that should be used before retrying the
// attempt. Will return error if the delay could not be determined.
RetryDelay(attempt int, opErr error) (time.Duration, error)
// GetRetryToken attempts to deduct the retry cost from the retry token pool.
// Returning the token release function, or error.
GetRetryToken(ctx context.Context, opErr error) (releaseToken func(error) error, err error)
// GetInitialToken returns the initial attempt token that can increment the
// retry token pool if the attempt is successful.
GetInitialToken() (releaseToken func(error) error)
}
// RetryerV2 is an interface to determine if a given error from an attempt
// should be retried, and if so what backoff delay to apply. The default
// implementation used by most services is the retry package's Standard type.
// Which contains basic retry logic using exponential backoff.
//
// RetryerV2 replaces the Retryer interface, deprecating the GetInitialToken
// method in favor of GetAttemptToken which takes a context, and can return an error.
//
// The SDK's retry package's Attempt middleware, and utilities will always
// wrap a Retryer as a RetryerV2. Delegating to GetInitialToken, only if
// GetAttemptToken is not implemented.
type RetryerV2 interface {
Retryer
// GetInitialToken returns the initial attempt token that can increment the
// retry token pool if the attempt is successful.
//
// Deprecated: This method does not provide a way to block using Context,
// nor can it return an error. Use RetryerV2, and GetAttemptToken instead.
GetInitialToken() (releaseToken func(error) error)
// GetAttemptToken returns the send token that can be used to rate limit
// attempt calls. Will be used by the SDK's retry package's Attempt
// middleware to get a send token prior to calling the temp and releasing
// the send token after the attempt has been made.
GetAttemptToken(context.Context) (func(error) error, error)
}
// NopRetryer provides a RequestRetryDecider implementation that will flag
// all attempt errors as not retryable, with a max attempts of 1.
type NopRetryer struct{}
// IsErrorRetryable returns false for all error values.
func (NopRetryer) IsErrorRetryable(error) bool { return false }
// MaxAttempts always returns 1 for the original attempt.
func (NopRetryer) MaxAttempts() int { return 1 }
// RetryDelay is not valid for the NopRetryer. Will always return error.
func (NopRetryer) RetryDelay(int, error) (time.Duration, error) {
return 0, fmt.Errorf("not retrying any attempt errors")
}
// GetRetryToken returns a stub function that does nothing.
func (NopRetryer) GetRetryToken(context.Context, error) (func(error) error, error) {
return nopReleaseToken, nil
}
// GetInitialToken returns a stub function that does nothing.
func (NopRetryer) GetInitialToken() func(error) error {
return nopReleaseToken
}
// GetAttemptToken returns a stub function that does nothing.
func (NopRetryer) GetAttemptToken(context.Context) (func(error) error, error) {
return nopReleaseToken, nil
}
func nopReleaseToken(error) error { return nil }
-14
View File
@@ -1,14 +0,0 @@
package aws
// ExecutionEnvironmentID is the AWS execution environment runtime identifier.
type ExecutionEnvironmentID string
// RuntimeEnvironment is a collection of values that are determined at runtime
// based on the environment that the SDK is executing in. Some of these values
// may or may not be present based on the executing environment and certain SDK
// configuration properties that drive whether these values are populated..
type RuntimeEnvironment struct {
EnvironmentIdentifier ExecutionEnvironmentID
Region string
EC2InstanceMetadataRegion string
}
-115
View File
@@ -1,115 +0,0 @@
package v4
import (
"strings"
"sync"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
)
func lookupKey(service, region string) string {
var s strings.Builder
s.Grow(len(region) + len(service) + 3)
s.WriteString(region)
s.WriteRune('/')
s.WriteString(service)
return s.String()
}
type derivedKey struct {
AccessKey string
Date time.Time
Credential []byte
}
type derivedKeyCache struct {
values map[string]derivedKey
mutex sync.RWMutex
}
func newDerivedKeyCache() derivedKeyCache {
return derivedKeyCache{
values: make(map[string]derivedKey),
}
}
func (s *derivedKeyCache) Get(credentials aws.Credentials, service, region string, signingTime SigningTime) []byte {
key := lookupKey(service, region)
s.mutex.RLock()
if cred, ok := s.get(key, credentials, signingTime.Time); ok {
s.mutex.RUnlock()
return cred
}
s.mutex.RUnlock()
s.mutex.Lock()
if cred, ok := s.get(key, credentials, signingTime.Time); ok {
s.mutex.Unlock()
return cred
}
cred := deriveKey(credentials.SecretAccessKey, service, region, signingTime)
entry := derivedKey{
AccessKey: credentials.AccessKeyID,
Date: signingTime.Time,
Credential: cred,
}
s.values[key] = entry
s.mutex.Unlock()
return cred
}
func (s *derivedKeyCache) get(key string, credentials aws.Credentials, signingTime time.Time) ([]byte, bool) {
cacheEntry, ok := s.retrieveFromCache(key)
if ok && cacheEntry.AccessKey == credentials.AccessKeyID && isSameDay(signingTime, cacheEntry.Date) {
return cacheEntry.Credential, true
}
return nil, false
}
func (s *derivedKeyCache) retrieveFromCache(key string) (derivedKey, bool) {
if v, ok := s.values[key]; ok {
return v, true
}
return derivedKey{}, false
}
// SigningKeyDeriver derives a signing key from a set of credentials
type SigningKeyDeriver struct {
cache derivedKeyCache
}
// NewSigningKeyDeriver returns a new SigningKeyDeriver
func NewSigningKeyDeriver() *SigningKeyDeriver {
return &SigningKeyDeriver{
cache: newDerivedKeyCache(),
}
}
// DeriveKey returns a derived signing key from the given credentials to be used with SigV4 signing.
func (k *SigningKeyDeriver) DeriveKey(credential aws.Credentials, service, region string, signingTime SigningTime) []byte {
return k.cache.Get(credential, service, region, signingTime)
}
func deriveKey(secret, service, region string, t SigningTime) []byte {
hmacDate := HMACSHA256([]byte("AWS4"+secret), []byte(t.ShortTimeFormat()))
hmacRegion := HMACSHA256(hmacDate, []byte(region))
hmacService := HMACSHA256(hmacRegion, []byte(service))
return HMACSHA256(hmacService, []byte("aws4_request"))
}
func isSameDay(x, y time.Time) bool {
xYear, xMonth, xDay := x.Date()
yYear, yMonth, yDay := y.Date()
if xYear != yYear {
return false
}
if xMonth != yMonth {
return false
}
return xDay == yDay
}
-40
View File
@@ -1,40 +0,0 @@
package v4
// Signature Version 4 (SigV4) Constants
const (
// EmptyStringSHA256 is the hex encoded sha256 value of an empty string
EmptyStringSHA256 = `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`
// UnsignedPayload indicates that the request payload body is unsigned
UnsignedPayload = "UNSIGNED-PAYLOAD"
// AmzAlgorithmKey indicates the signing algorithm
AmzAlgorithmKey = "X-Amz-Algorithm"
// AmzSecurityTokenKey indicates the security token to be used with temporary credentials
AmzSecurityTokenKey = "X-Amz-Security-Token"
// AmzDateKey is the UTC timestamp for the request in the format YYYYMMDD'T'HHMMSS'Z'
AmzDateKey = "X-Amz-Date"
// AmzCredentialKey is the access key ID and credential scope
AmzCredentialKey = "X-Amz-Credential"
// AmzSignedHeadersKey is the set of headers signed for the request
AmzSignedHeadersKey = "X-Amz-SignedHeaders"
// AmzSignatureKey is the query parameter to store the SigV4 signature
AmzSignatureKey = "X-Amz-Signature"
// TimeFormat is the time format to be used in the X-Amz-Date header or query parameter
TimeFormat = "20060102T150405Z"
// ShortTimeFormat is the shorten time format used in the credential scope
ShortTimeFormat = "20060102"
// ContentSHAKey is the SHA256 of request body
ContentSHAKey = "X-Amz-Content-Sha256"
// StreamingEventsPayload indicates that the request payload body is a signed event stream.
StreamingEventsPayload = "STREAMING-AWS4-HMAC-SHA256-EVENTS"
)
Loaded 100 of 1614 files, more files were not shown because too many files have changed in this diff. Show more