Compare commits

...
95 Commits
Author SHA1 Message Date
Bill Oley fed723614b ifixes #219 - create scripts to help with running and testing multiple instances 2021-10-21 16:35:33 -04:00
dependabot[bot] 729dca23ba Bump httpclient from 4.5.6 to 4.5.13 (#215)
Bumps httpclient from 4.5.6 to 4.5.13.

---
updated-dependencies:
- dependency-name: org.apache.httpcomponents:httpclient
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-03 23:49:36 -04:00
Bill Oley 104d20ad1b Bump version to 1.3.3-SNAPSHOT 2020-11-06 13:30:04 -05:00
Bill Oley f0f005f8fa Release version 1.3.2 2020-11-06 13:23:36 -05:00
Bill Oley 8593218b89 Logging fixes 2020-11-06 12:24:22 -05:00
Bill Oley cbfe6a439f closes #213 Release reference counted netty objects to avoid memory errors 2020-11-06 11:50:24 -05:00
Bill Oley 455616d568 closes #213 Release reference counted netty objects to avoid memory errors 2020-11-05 14:17:01 -05:00
Bill Oley 9d3172cc75 Close objects when TcpClient has problem connecting to Timely 2020-11-04 16:06:08 -05:00
Bill Oley 7aa7b0da2a Remove entries from metricMap when metric is no longer cached on a particular server 2020-11-04 16:04:52 -05:00
Bill Oley 6c61861d83 Fix NPE in BalancedMetricResolver.rebalance where a metric exists in metricMap but not in metricToHostMap 2020-10-29 16:49:55 -04:00
Bill Oley 91882ba363 closes #212 Use writeLock for DataStoreCache.ageOffGorillaStores to avoid ConcurrentModificationException 2020-10-28 15:58:33 -04:00
Bill Oley ed61b5f305 Fix css inconsistent rendering in grafana datasource QueryEditor, remove overridden/unusd css 2020-10-27 14:10:58 -04:00
Bill Oley a083c2d632 Fix oauth issues - intersect TimelyUser auths with Accumulo auths when creating from oauthToken, allow X.509 (PEM) jwt signature check key, increase size of httpHeader max size 2020-10-23 14:41:46 -04:00
Jonathan SteinandJonathan Stein 2891794174 Fixing Nondeterministic Tests (#211)
* Fix MetricAdapterTest and QueryResponseTest flakiness

* change MetricAdapterTest and QueryResponseTest flakiness fix by defining Json key order

* Removed .swp file

* Fix SearchLookupResponse testResponse1

* Fixed flaky AuthCacheTest

Co-authored-by: Jonathan Stein <[email protected]>
2020-10-19 14:01:17 -04:00
Bill Oley c99b2471cf Bump version to 1.3.2-SNAPSHOT 2020-10-14 14:58:07 -04:00
Bill Oley bb479a58d7 Release version 1.3.1 2020-10-14 14:54:59 -04:00
Bill Oley ff37d40801 datasource logging fix, combine datasource golang code for readability 2020-10-14 14:46:09 -04:00
Bill Oley a7d980975e closes #210 Only reload cert and CA when changed
Only reload cert and CA when changed
2020-10-14 13:12:11 -04:00
dependabot[bot] 30f5a0c1b2 Bump junit from 4.12 to 4.13.1 (#209)
Bumps [junit](https://github.com/junit-team/junit4) from 4.12 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-13 00:54:58 -04:00
Bill Oley 2f9e6ca2d7 Bump version to 1.3.1-SNAPSHOT 2020-09-25 13:04:54 -04:00
Bill Oley 954528a460 Release version 1.3.0 2020-09-25 12:39:33 -04:00
Bill Oley 0f7b514b30 Update authentication configuration and error handling, Grafana depenancies to 7.2.0
Grafana depenancies to 7.2.0
2020-09-25 12:19:08 -04:00
Bill Oley a6ca46e1c1 Rewrite grafana plugin for grafana 7 using typescript, react. Use new backend plugin feature to enable alerts
More work - has errors

more work

more work

Add addOauthPassThru setting

CS fixes, OAuthPassThru setting, Alias fix, Auto Focus on new tag

Logging cleanup and make aliases work

Delete old version of plugin

remove extra files

Accept JWT authentication and test fixes

Accept JWT authentication

Fix assembly

Add JWT settings to sample configs
2020-09-15 16:26:29 -04:00
Bill Oley 487d1d4a5c Use MetaCache for suggest api and implement tagk and tagv types of suggest 2020-09-15 16:26:18 -04:00
Christopher Tubbs e3c26f815c Merge pull request #206 from NationalSecurityAgency/dependabot/maven/commons-beanutils-commons-beanutils-1.9.4
Bump commons-beanutils from 1.9.3 to 1.9.4
2020-07-02 17:16:58 -04:00
Christopher Tubbs bbc1946204 Update plugins for newer JDKs
* Bump formatter and impsort to latest to stay current
* Bump spotbugs to support up to at least JDK 14
2020-07-02 17:09:06 -04:00
dependabot[bot] 991412f99b Bump commons-beanutils from 1.9.3 to 1.9.4
Bumps commons-beanutils from 1.9.3 to 1.9.4.

Signed-off-by: dependabot[bot] <[email protected]>
2020-07-02 20:44:34 +00:00
Christopher Tubbs 5428e93cb5 Merge pull request #205 from NationalSecurityAgency/dependabot/maven/version.log4j-2.13.3
Bump version.log4j from 2.11.2 to 2.13.3
2020-07-02 16:44:06 -04:00
dependabot[bot] c18aae5a89 Bump version.log4j from 2.11.2 to 2.13.3
Bumps `version.log4j` from 2.11.2 to 2.13.3.

Updates `log4j-api` from 2.11.2 to 2.13.3

Updates `log4j-core` from 2.11.2 to 2.13.3

Updates `log4j-slf4j-impl` from 2.11.2 to 2.13.3

Signed-off-by: dependabot[bot] <[email protected]>
2020-07-01 18:20:00 +00:00
Bill Oley 6972c46a1a Merge pull request #204 from matthpeterson/issue-203
Closes issue #203 - avoid calling sizeInBytes if it's not expected to exist
2020-04-17 10:42:14 -04:00
matthpeterson 4d7bee7566 Closes issue #203 - Stop calling sizeInBytes if it's not expected to exist 2020-04-16 15:17:08 +00:00
Bill Oley 4b11b3d50f closes #201 Create balancer assignment file if it doesn't exist 2020-04-10 10:27:18 -04:00
Bill Oley 84b9856ec7 closes #199 Fix issues when using a generated (self-signed) server certificate 2020-04-10 10:20:25 -04:00
Bill Oley c9548a9b9a closes #198 Cleaner shutdown of resources for Timely server, balancer, grafana-auth 2020-04-10 10:01:47 -04:00
Bill Oley aec7bd4c38 closes #197 Fix scan range in MetaCacheImpl 2020-04-09 11:58:30 -04:00
Bill Oley e2b8b643f2 closes #200 Initialize non-cached metrics stored in zookeeper 2020-04-09 11:32:03 -04:00
Bill Oley c19e1d8b2f Set version to 1.2.8-SNAPSHOT 2020-02-21 10:20:48 -05:00
Bill Oley cb73e8e75c Tag version 1.2.7 2020-02-21 10:02:31 -05:00
Bill Oley 4c508c1a7d Merge pull request #195 from dtspence/TIMELY-194
Fixed #194, corrected datasource.js to look at wildcard tag value
2020-02-21 08:55:37 -05:00
dtspence c6c2c0d41e Fixed #194, corrected datasource.js to look at wildcard tag value
The alignment process to match tag query results with the metric definition
looks at the tag k/v pairs and makes a special consideration for a wildcard tag value.
The change adjusts datasource.js so that the '.*' notation will match a query result
and metric definition.
2020-02-05 19:56:03 +00:00
Christopher Tubbs a24cef78b3 Update jackson (multiple CVEs) (#193) 2019-11-07 23:05:38 -05:00
Bill Oley 08f99f6540 Set version to 1.2.7-SNAPSHOT 2019-10-30 12:33:49 -04:00
Bill Oley be11da7d15 Tag version 1.2.6 2019-10-30 12:30:33 -04:00
Bill Oley 4bc5753328 fixes #192 Thread deadlock in BalancedMetricResolver 2019-10-29 12:24:10 -04:00
Bill Oley 2ccb89a06d re #191 Correct typos 2019-10-18 09:16:03 -04:00
Bill Oley dfc3840ad4 re #191 Delete cached metrics when they are no longer being updated 2019-10-17 17:49:24 -04:00
Bill Oley 6235034d8b BalancedMetricResolver in the non-leader balancer should read assignments at intervals 2019-10-17 17:49:24 -04:00
Bill Oley d8de6ac06f Make zookeeper timeout configurable 2019-10-17 17:49:24 -04:00
Christopher Tubbs f381946abe Merge pull request #190 from NationalSecurityAgency/dependabot/maven/io.netty-netty-all-4.1.42.Final
Bump netty-all from 4.1.35.Final to 4.1.42.Final
2019-10-17 09:35:13 -07:00
dependabot[bot] 48bbe6a561 Bump netty-all from 4.1.35.Final to 4.1.42.Final
Bumps [netty-all](https://github.com/netty/netty) from 4.1.35.Final to 4.1.42.Final.
- [Release notes](https://github.com/netty/netty/releases)
- [Commits](https://github.com/netty/netty/compare/netty-4.1.35.Final...netty-4.1.42.Final)

Signed-off-by: dependabot[bot] <[email protected]>
2019-10-11 18:58:57 +00:00
Christopher Tubbs 0f6110ea59 Update jackson to 2.9.10
* Fix CVE-2019-16335
* Fix CVE-2019-14540
2019-09-26 23:40:08 -04:00
Bill Oley 461b5e7e78 Pom version to 1.2.6-SNAPSHOT 2019-08-22 12:17:45 -04:00
Bill Oley 4ec7d94c28 Tag version 1.2.5 2019-08-22 12:12:49 -04:00
Bill Oley 77e910ae06 Use BufferedOutputWriter instead of PrintWriter, pass Exceptions instead of having to check error condition 2019-08-22 10:09:27 -04:00
Bill Oley a4998af887 Move jackson-databind version to the properties section 2019-08-08 13:50:08 -04:00
Bill Oley 2858549ede Merge pull request #188 from ctubbsii/jdk11-build-support
Bump versions / permit build with JDK11
2019-08-08 13:35:54 -04:00
Christopher Tubbs 57f0e1408f Bump versions / permit build with JDK11
Support builds on newer JDKs
* Update tycho-compiler-jdt in analytics/pom.xml
* Ignore spotbugs RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE for JDK11
* Bump several plugins to latest (maven-compiler-plugin, etc.)
* Add profile for using -release flag in Eclipse and with JDK9 and later

Fix recently added warnings
* Add try-with-resources for TreeCache objects to remove recent warnings
* Suppress generics warning in MetricCompactionStrategy
* Update TabletMetadataConsole to avoid deprecated Accumulo
  ClientConfiguration API

Other
* Bump jackson-databind for CVE-2019-14439, CVE-2019-14379
* Bump hadoop 2.7 to latest in series (2.7.3->2.7.7) for build
2019-08-06 16:14:13 -04:00
Bill Oley 151adc025a Merge pull request #185 from dtspence/TIMELY-184
Fixes #184 Metric compaction strategy and summary implementation
2019-08-06 15:15:45 -04:00
Bill Oley 5abd9d7200 closes #187 Prevent deadlock in balancer distributed read/write lock 2019-08-06 12:00:57 -04:00
Bill Oley fa2e211c72 closes #186 Make balancer websocket incomingBufferSize its own setting 2019-08-06 11:47:36 -04:00
dtspence a21438976e TIMELY-184: Metric compaction strategy and summary implementation. 2019-08-01 14:05:35 +00:00
Christopher Tubbs ba37047f3a Update jackson-databind (CVE-2019-12814) 2019-07-23 20:38:46 -04:00
Bill Oley 9305470c69 re #183 Check tagKey not null or blank in Meta.toKeys 2019-06-19 09:08:20 -04:00
Bill Oley 200233542e re #183 Add timestamps to meta tags to allow proper age off 2019-06-17 15:36:51 -04:00
Bill Oley 3cd78dc624 Merge branch 'drewfarris-config-fixup' 2019-05-30 12:55:35 -04:00
Bill Oley 9027cac6e3 Merge branch 'config-fixup' of https://github.com/drewfarris/timely into drewfarris-config-fixup
Conflicts:
	server/conf/timely-docker.yml
	server/conf/timely-standalone.yml
	server/conf/timely.yml
2019-05-30 12:50:31 -04:00
Bill Oley 3663b4e6d5 Pom version to 1.2.5-SNAPSHOT 2019-05-30 12:38:42 -04:00
Bill Oley 64973d0674 Merge pull request #182 from ctubbsii/fix-jackson
Fix most compiler warnings and many other issues
2019-05-30 12:25:00 -04:00
Christopher Tubbs 5512306f1e Fix most compiler warnings and other issues
Build:
* Fix netty-tcnative-boringssl-static version specs in pom.xml
* Bump the version of accumulo to latest release (1.9.3)
* Update jackson to 2.9.9 (CVE-2019-12086)

Warnings:
* Remove all use of deprecated netty stuffs and use their replacements
  in the netty APIs
* Add missing generics parameters everywhere
* Remove unused code (unused private fields, such as loggers)
* Remove standard javadoc tags that were missing descriptions (these do
  not add value to the javadocs, since default `@param`, `@throws`, and
  `@return` tags are provided by javadoc)
* Add missing serialVersionUid fields to Serializable classes
* Fix a resource leak warning about not closing ScannerOptionsHelper
  (NOTE: 3 other resource leak warnings exist, but did not attempt to
  fix those, since they were non-trivial)
* Remove deprecated annotations from hibernate and replace with their
  non-deprecated counterparts from javax.validation.constraints
* Remove use of deprecated Accumulo ClientConfiguration constructor

Other:
* Use lowercase header names, for HTTP/2 compatibility (and to stop
  incorrectly assuming case-sensitivity for HTTP/1)
* Narrow the scope of necessary `@SuppressWarnings("unchecked")`
  occurrences to a variable instead of suppressing for an entire method or
  an entire class
* IDE added some missing overrides annotations
* (style consistency) Replace tabs with spaces
* Set log4j2 root logger level for tests to WARN instead of TRACE
2019-05-24 12:12:00 -04:00
Bill Oley d35ae08188 Tag version 1.2.4 2019-05-22 17:14:42 -04:00
Bill Oley e448657afb Correction in websocket pipeline from Netty API change 2019-05-22 16:49:11 -04:00
Bill Oley 1dcd893711 Add age off iterator for meta table
Add age off iterator for meta table
2019-05-22 16:49:08 -04:00
Bill Oley 409f5375d1 Change pom version to 1.2.4-SNAPSHOT 2019-05-22 16:48:09 -04:00
Bill Oley c5df8139ca Correct typos 2019-05-22 16:46:55 -04:00
Bill Oley dcd23d4fc8 fixes #181 Use netty-tcnative-boringssl-static 2019-04-18 11:56:54 -04:00
Bill Oley cc7326fc19 Tag version 1.2.2 2019-04-12 11:24:53 -04:00
Bill Oley 0d2b6e61c8 Correct typos 2019-04-12 10:55:04 -04:00
Bill Oley b7cd8765c5 Correct typos and minor changed to MetaCache 2019-04-10 15:31:53 -04:00
Bill Oley a9d27d5b1b re #179 Allow AuthCache to periodically refresh credentials, cache meta table up to a configurable number of tags 2019-04-08 10:50:14 -04:00
Bill Oley 7d67eb218d Update netty-tcnative dependency, remove x509 principal extractor from AuthenticationService 2019-04-03 15:52:19 -04:00
Bill Oley 5fb7e3dd2b Bump pom versions to 1.2.2-SNAPSHOT 2019-04-02 12:57:22 -04:00
Bill Oley 5a89861cec Tag version 1.2.1 2019-04-02 12:53:04 -04:00
Bill Oley a3bc9a01ee Update dependency versions
Update dependencies

Update dependency versions
2019-04-02 12:43:36 -04:00
Bill Oley dba494b06d Bump pom versions to 1.2.1-SNAPSHOT 2019-02-25 15:41:31 -05:00
Bill Oley 5826884c3c Tag version 1.2.0 2019-02-25 15:11:57 -05:00
Bill Oley f94c731bbc Remove unused modeule directory introduced in error 2019-02-25 15:11:12 -05:00
Bill Oley 4e042bcb94 Allow all http methods in grafana auth proxy, disable cookies in HttpPools 2019-02-22 16:17:36 -05:00
Bill Oley ab7f499134 closes #180 Allow a configurable default visibility for metrics 2019-02-21 15:33:25 -05:00
Bill Oley c098f9f3e4 re #179 Create an authentication app for grafana that uses Timely's authentication framework and grafana's auth-proxy
re #179 Create authentication proxy for grafana
2019-02-21 13:28:12 -05:00
Bill Oley 94cfcbc0c4 Allow authentication provider to return roles, check users against requiredRoles and requiredAuths for access
index on master: 674e1b0 Improve synchronization in AuthenticationService

Check validated users against requiredRoles and requiredAuths

Conflicts:
	server/conf/security.xml
	server/src/main/java/timely/auth/AuthenticationService.java

More work
2019-02-15 11:24:07 -05:00
Bill Oley 674e1b0ab5 Improve synchronization in AuthenticationService 2019-02-08 16:53:35 -05:00
Bill Oley 4eedb84547 Add synchronization to AuthenticationService to prevent simultaneous authentications of the same user by multiple threads 2019-02-07 17:17:08 -05:00
Bill Oley bef86eac37 Fix websocket proxying with a client certificate, add ability to configure client cert in python websocket client 2019-02-07 17:16:00 -05:00
Bill Oley 27bb16e403 re #178 Allow proxying of http and ws requests
More work

Correct errors
2019-02-05 11:41:35 -05:00
Drew Farris e2bbe4e212 Minor fix-ups to default configuration files.
. Replace ssl with server-ssl in timely-standalone.yml and timely-docker.yml
  . Add a value for cache.enabled in timely.yml
2019-01-19 22:07:07 -05:00
353 changed files with 240984 additions and 24394 deletions

No files matched your search

+257 -256
View File
@@ -1,256 +1,257 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>timely-analytics</artifactId>
<!--
Execute "mvn clean package -Pbuild-jar"
to build a jar file out of this project!
How to use the Flink Quickstart pom:
a) Adding new dependencies:
You can add dependencies to the list below.
Please check if the maven-shade-plugin below is filtering out your dependency
and remove the exclude from there.
b) Build a jar for running on the cluster:
There are two options for creating a jar from this project
b.1) "mvn clean package" -> this will create a fat jar which contains all
dependencies necessary for running the jar created by this pom in a cluster.
The "maven-shade-plugin" excludes everything that is provided on a running Flink cluster.
b.2) "mvn clean package -Pbuild-jar" -> This will also create a fat-jar, but with much
nicer dependency exclusion handling. This approach is preferred and leads to
much cleaner jar files.
-->
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-client</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerId>jdt</compilerId>
</configuration>
<dependencies>
<dependency>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-compiler-jdt</artifactId>
<version>0.21.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
<!-- We use the maven-shade plugin to create a fat jar that contains all dependencies
except flink and it's transitive dependencies. The resulting fat-jar can be executed
on a cluster. Change the value of Program-Class if your program entry point changes. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
<configuration>
<artifactSet>
<excludes>
<!-- This list contains all dependencies of flink-dist
Everything else will be packaged into the fat-jar
-->
<exclude>org.apache.flink:flink-annotations</exclude>
<exclude>org.apache.flink:flink-shaded-hadoop1_2.11</exclude>
<exclude>org.apache.flink:flink-shaded-hadoop2</exclude>
<exclude>org.apache.flink:flink-shaded-curator-recipes</exclude>
<exclude>org.apache.flink:flink-core</exclude>
<exclude>org.apache.flink:flink-java</exclude>
<exclude>org.apache.flink:flink-scala_2.11</exclude>
<exclude>org.apache.flink:flink-runtime_2.11</exclude>
<exclude>org.apache.flink:flink-optimizer_2.11</exclude>
<exclude>org.apache.flink:flink-clients_2.11</exclude>
<exclude>org.apache.flink:flink-avro_2.11</exclude>
<exclude>org.apache.flink:flink-examples-batch_2.11</exclude>
<exclude>org.apache.flink:flink-examples-streaming_2.11</exclude>
<exclude>org.apache.flink:flink-streaming-java_2.11</exclude>
<!-- Also exclude very big transitive dependencies of Flink
WARNING: You have to remove these excludes if your code relies on other
versions of these dependencies.
-->
<exclude>org.scala-lang:scala-library</exclude>
<exclude>org.scala-lang:scala-compiler</exclude>
<exclude>org.scala-lang:scala-reflect</exclude>
<exclude>com.typesafe.akka:akka-actor_*</exclude>
<exclude>com.typesafe.akka:akka-remote_*</exclude>
<exclude>com.typesafe.akka:akka-slf4j_*</exclude>
<exclude>io.netty:netty-all</exclude>
<exclude>io.netty:netty</exclude>
<exclude>commons-fileupload:commons-fileupload</exclude>
<exclude>org.apache.avro:avro</exclude>
<exclude>commons-collections:commons-collections</exclude>
<exclude>org.codehaus.jackson:jackson-core-asl</exclude>
<exclude>org.codehaus.jackson:jackson-mapper-asl</exclude>
<exclude>com.thoughtworks.paranamer:paranamer</exclude>
<exclude>org.xerial.snappy:snappy-java</exclude>
<exclude>org.apache.commons:commons-compress</exclude>
<exclude>org.tukaani:xz</exclude>
<exclude>com.esotericsoftware.kryo:kryo</exclude>
<exclude>com.esotericsoftware.minlog:minlog</exclude>
<exclude>org.objenesis:objenesis</exclude>
<exclude>com.twitter:chill_*</exclude>
<exclude>com.twitter:chill-java</exclude>
<exclude>commons-lang:commons-lang</exclude>
<exclude>junit:junit</exclude>
<exclude>org.apache.commons:commons-lang3</exclude>
<exclude>org.slf4j:slf4j-api</exclude>
<exclude>org.slf4j:slf4j-log4j12</exclude>
<exclude>log4j:log4j</exclude>
<exclude>org.apache.commons:commons-math</exclude>
<exclude>org.apache.sling:org.apache.sling.commons.json</exclude>
<exclude>commons-logging:commons-logging</exclude>
<exclude>commons-codec:commons-codec</exclude>
<exclude>com.fasterxml.jackson.core:jackson-core</exclude>
<exclude>com.fasterxml.jackson.core:jackson-databind</exclude>
<exclude>com.fasterxml.jackson.core:jackson-annotations</exclude>
<exclude>stax:stax-api</exclude>
<exclude>com.typesafe:config</exclude>
<exclude>org.uncommons.maths:uncommons-maths</exclude>
<exclude>com.github.scopt:scopt_*</exclude>
<exclude>commons-io:commons-io</exclude>
<exclude>commons-cli:commons-cli</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>org.apache.flink:*</artifact>
<excludes>
<!-- exclude shaded google but include shaded curator -->
<exclude>org/apache/flink/shaded/com/**</exclude>
<exclude>web-docs/**</exclude>
</excludes>
</filter>
<filter>
<!-- Do not copy the signatures in the META-INF folder.
Otherwise, this might cause SecurityExceptions when using the JAR.
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines.
This will add a Main-Class entry to the manifest file -->
<!--
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>jar.StreamingJob</mainClass>
</transformer>
</transformers>
-->
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>buildnumber-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<!-- Profile for packaging correct JAR files -->
<id>build-jar</id>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- disable the exclusion rules -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
<configuration>
<artifactSet>
<excludes combine.self="override" />
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>timely-analytics</artifactId>
<!--
Execute "mvn clean package -Pbuild-jar"
to build a jar file out of this project!
How to use the Flink Quickstart pom:
a) Adding new dependencies:
You can add dependencies to the list below.
Please check if the maven-shade-plugin below is filtering out your dependency
and remove the exclude from there.
b) Build a jar for running on the cluster:
There are two options for creating a jar from this project
b.1) "mvn clean package" -> this will create a fat jar which contains all
dependencies necessary for running the jar created by this pom in a cluster.
The "maven-shade-plugin" excludes everything that is provided on a running Flink cluster.
b.2) "mvn clean package -Pbuild-jar" -> This will also create a fat-jar, but with much
nicer dependency exclusion handling. This approach is preferred and leads to
much cleaner jar files.
-->
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-client</artifactId>
<exclusions>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerId>jdt</compilerId>
</configuration>
<dependencies>
<dependency>
<groupId>org.eclipse.tycho</groupId>
<artifactId>tycho-compiler-jdt</artifactId>
<version>1.4.0</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
</plugin>
<!-- We use the maven-shade plugin to create a fat jar that contains all dependencies
except flink and it's transitive dependencies. The resulting fat-jar can be executed
on a cluster. Change the value of Program-Class if your program entry point changes. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
<configuration>
<artifactSet>
<excludes>
<!-- This list contains all dependencies of flink-dist
Everything else will be packaged into the fat-jar
-->
<exclude>org.apache.flink:flink-annotations</exclude>
<exclude>org.apache.flink:flink-shaded-hadoop1_2.11</exclude>
<exclude>org.apache.flink:flink-shaded-hadoop2</exclude>
<exclude>org.apache.flink:flink-shaded-curator-recipes</exclude>
<exclude>org.apache.flink:flink-core</exclude>
<exclude>org.apache.flink:flink-java</exclude>
<exclude>org.apache.flink:flink-scala_2.11</exclude>
<exclude>org.apache.flink:flink-runtime_2.11</exclude>
<exclude>org.apache.flink:flink-optimizer_2.11</exclude>
<exclude>org.apache.flink:flink-clients_2.11</exclude>
<exclude>org.apache.flink:flink-avro_2.11</exclude>
<exclude>org.apache.flink:flink-examples-batch_2.11</exclude>
<exclude>org.apache.flink:flink-examples-streaming_2.11</exclude>
<exclude>org.apache.flink:flink-streaming-java_2.11</exclude>
<!-- Also exclude very big transitive dependencies of Flink
WARNING: You have to remove these excludes if your code relies on other
versions of these dependencies.
-->
<exclude>org.scala-lang:scala-library</exclude>
<exclude>org.scala-lang:scala-compiler</exclude>
<exclude>org.scala-lang:scala-reflect</exclude>
<exclude>com.typesafe.akka:akka-actor_*</exclude>
<exclude>com.typesafe.akka:akka-remote_*</exclude>
<exclude>com.typesafe.akka:akka-slf4j_*</exclude>
<exclude>io.netty:netty-all</exclude>
<exclude>io.netty:netty</exclude>
<exclude>commons-fileupload:commons-fileupload</exclude>
<exclude>org.apache.avro:avro</exclude>
<exclude>commons-collections:commons-collections</exclude>
<exclude>org.codehaus.jackson:jackson-core-asl</exclude>
<exclude>org.codehaus.jackson:jackson-mapper-asl</exclude>
<exclude>com.thoughtworks.paranamer:paranamer</exclude>
<exclude>org.xerial.snappy:snappy-java</exclude>
<exclude>org.apache.commons:commons-compress</exclude>
<exclude>org.tukaani:xz</exclude>
<exclude>com.esotericsoftware.kryo:kryo</exclude>
<exclude>com.esotericsoftware.minlog:minlog</exclude>
<exclude>org.objenesis:objenesis</exclude>
<exclude>com.twitter:chill_*</exclude>
<exclude>com.twitter:chill-java</exclude>
<exclude>commons-lang:commons-lang</exclude>
<exclude>junit:junit</exclude>
<exclude>org.apache.commons:commons-lang3</exclude>
<exclude>org.slf4j:slf4j-api</exclude>
<exclude>org.slf4j:slf4j-log4j12</exclude>
<exclude>log4j:log4j</exclude>
<exclude>org.apache.commons:commons-math</exclude>
<exclude>org.apache.sling:org.apache.sling.commons.json</exclude>
<exclude>commons-logging:commons-logging</exclude>
<exclude>commons-codec:commons-codec</exclude>
<exclude>com.fasterxml.jackson.core:jackson-core</exclude>
<exclude>com.fasterxml.jackson.core:jackson-databind</exclude>
<exclude>com.fasterxml.jackson.core:jackson-annotations</exclude>
<exclude>stax:stax-api</exclude>
<exclude>com.typesafe:config</exclude>
<exclude>org.uncommons.maths:uncommons-maths</exclude>
<exclude>com.github.scopt:scopt_*</exclude>
<exclude>commons-io:commons-io</exclude>
<exclude>commons-cli:commons-cli</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>org.apache.flink:*</artifact>
<excludes>
<!-- exclude shaded google but include shaded curator -->
<exclude>org/apache/flink/shaded/com/**</exclude>
<exclude>web-docs/**</exclude>
</excludes>
</filter>
<filter>
<!-- Do not copy the signatures in the META-INF folder.
Otherwise, this might cause SecurityExceptions when using the JAR.
-->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines.
This will add a Main-Class entry to the manifest file -->
<!--
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>jar.StreamingJob</mainClass>
</transformer>
</transformers>
-->
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>buildnumber-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<!-- Profile for packaging correct JAR files -->
<id>build-jar</id>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- disable the exclusion rules -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
<configuration>
<artifactSet>
<excludes combine.self="override" />
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
@@ -66,7 +66,7 @@ public class SubscriptionSource extends RichSourceFunction<MetricResponse> imple
LOG.info("Opening summarization job.");
super.open(parameters);
client = new WebSocketSubscriptionClient(jp.getTimelyHostname(), jp.getTimelyHttpsPort(), jp.getTimelyWssPort(),
jp.isDoLogin(), jp.getTimelyUsername(), jp.getTimelyPassword(), jp.getKeyStoreFile(),
jp.isClientAuth(), jp.isDoLogin(), jp.getTimelyUsername(), jp.getTimelyPassword(), jp.getKeyStoreFile(),
jp.getKeyStoreType(), jp.getKeyStorePass(), jp.getTrustStoreFile(), jp.getTrustStoreType(),
jp.getTrustStorePass(), jp.isHostVerificationEnabled(), jp.getBufferSize());
}
@@ -67,7 +67,7 @@ public class SummarizationJob {
})
.name("Metric Windows")
.addSink(
new SocketClientSink<MetricHistogram>(jp.getTimelyHostname(), jp.getTimelyTcpPort(),
new SocketClientSink<MetricHistogram>(jp.getTimelyHostname(), jp.getTimelyTcpPort(),
new MetricHistogram(), 2, true) {
private final Logger LOG = LoggerFactory
@@ -18,6 +18,7 @@ public class SummarizationJobParameters extends GlobalJobParameters implements S
private final int timelyTcpPort;
private final int timelyHttpsPort;
private final int timelyWssPort;
private final boolean clientAuth;
private final boolean doLogin;
private final String timelyUsername;
private final String timelyPassword;
@@ -40,6 +41,7 @@ public class SummarizationJobParameters extends GlobalJobParameters implements S
timelyTcpPort = params.getInt("timelyTcpPort", 4241);
timelyHttpsPort = params.getInt("timelyHttpsPort", 4242);
timelyWssPort = params.getInt("timelyWssPort", 4243);
clientAuth = params.getBoolean("clientAuth", false);
doLogin = params.getBoolean("doLogin", false);
timelyUsername = params.get("timelyUsername", null);
timelyPassword = params.get("timelyPassword", null);
@@ -69,6 +71,7 @@ public class SummarizationJobParameters extends GlobalJobParameters implements S
timelyTcpPort = conf.getInteger("timelyTcpPort", 4241);
timelyHttpsPort = conf.getInteger("timelyHttpsPort", 4242);
timelyWssPort = conf.getInteger("timelyWssPort", 4243);
clientAuth = conf.getBoolean("clientAuth", false);
doLogin = conf.getBoolean("doLogin", false);
timelyUsername = conf.getString("timelyUsername", null);
timelyPassword = conf.getString("timelyPassword", null);
@@ -112,6 +115,10 @@ public class SummarizationJobParameters extends GlobalJobParameters implements S
return timelyWssPort;
}
public boolean isClientAuth() {
return clientAuth;
}
public boolean isDoLogin() {
return doLogin;
}
+4 -5
View File
@@ -1,4 +1,4 @@
#!/bin/bash -vx
#!/bin/bash
if [[ `uname` == "Darwin" ]]; then
THIS_SCRIPT=`python -c 'import os,sys; print os.path.realpath(sys.argv[1])' $0`
@@ -27,15 +27,14 @@ fi
mkdir -p ${NATIVE_DIR}
pushd ${BASE_DIR}/bin
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-tcnative*.jar META-INF/native/libnetty_tcnative.${TCNATIVE_SUFFIX}
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-transport-native-epoll*.jar META-INF/native/libnetty_transport_native_epoll_x86_64.${TCNATIVE_SUFFIX}
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-tcnative-boringssl-static*.jar META-INF/native/libnetty_tcnative_linux_x86_64.${TCNATIVE_SUFFIX}
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-all*.jar META-INF/native/libnetty_transport_native_epoll_x86_64.${TCNATIVE_SUFFIX}
popd
export CLASSPATH="${CONF_DIR}:${LIB_DIR}/*:${HADOOP_CONF_DIR}"
JVM_ARGS="-Xmx1G -Xms1G -Dio.netty.eventLoopThreads=${NUM_SERVER_THREADS}"
JVM_ARGS="-Xmx1G -Xms1G -Dio.netty.eventLoopThreads=${NUM_SERVER_THREADS} -Dlog4j.configurationFile=${CONF_DIR}/log4j2-spring.xml"
JVM_ARGS="${JVM_ARGS} -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector"
JVM_ARGS="${JVM_ARGS} -Djava.library.path=${NATIVE_DIR}"
JVM_ARGS="${JVM_ARGS} -Djava.library.path=${NATIVE_DIR}/libnetty-${TCNATIVE_SUFFIX}"
#JVM_ARGS="${JVM_ARGS} -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
+39 -17
View File
@@ -11,21 +11,33 @@
<beans:bean id="x509PrincipalExtractor" class="org.springframework.security.web.authentication.preauth.x509.SubjectDnX509PrincipalExtractor">
<beans:property name="subjectDnRegex" value="CN=(.*?)(?:,|$)"/>
</beans:bean>
</beans:bean>
<beans:bean id="timelyUserDetailsService" class="timely.auth.UserDetailsService">
<beans:property name="users">
<util:map key-type="java.lang.String">
<beans:entry key="example.com">
<util:list>
<beans:value>D</beans:value>
<beans:value>E</beans:value>
<beans:value>F</beans:value>
</util:list>
</beans:entry>
</util:map>
</beans:property>
</beans:bean>
<beans:bean id="timelyUserDetailsService" class="timely.auth.UserDetailsService">
<beans:property name="users">
<util:list>
<beans:bean class="timely.auth.TimelyUser">
<beans:constructor-arg name="subjectDn">
<beans:value>CN=example.com</beans:value>
</beans:constructor-arg>
<beans:constructor-arg name="auths">
<util:list>
<beans:value>D</beans:value>
<beans:value>E</beans:value>
<beans:value>F</beans:value>
</util:list>
</beans:constructor-arg>
<beans:constructor-arg name="roles">
<util:list>
<beans:value>G</beans:value>
<beans:value>H</beans:value>
<beans:value>I</beans:value>
</util:list>
</beans:constructor-arg>
</beans:bean>
</util:list>
</beans:property>
</beans:bean>
<beans:bean id="x509AuthorizationProvider" class="org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationProvider">
<beans:property name="preAuthenticatedUserDetailsService" ref="timelyUserDetailsService" />
@@ -34,9 +46,12 @@
<user-service id="usernamePasswordService">
<user name="test" password="test1" authorities="A,B,C" />
</user-service>
<beans:bean id="UserNamePasswordAuthenticationProvider" class="org.springframework.security.authentication.dao.DaoAuthenticationProvider">
<beans:bean id="noOpPasswordEncoder" class="org.springframework.security.crypto.password.NoOpPasswordEncoder"/>
<beans:bean id="UserNamePasswordAuthenticationProvider" class="org.springframework.security.authentication.dao.DaoAuthenticationProvider">
<beans:property name="userDetailsService" ref="usernamePasswordService"/>
<beans:property name="passwordEncoder" ref="noOpPasswordEncoder"/>
</beans:bean>
<authentication-manager alias="authenticationManager">
@@ -45,5 +60,12 @@
<!-- Basic authorization provider, validates provided username and password -->
<authentication-provider ref="UserNamePasswordAuthenticationProvider" />
</authentication-manager>
<util:list id="requiredRoles" value-type="java.lang.String">
<beans:value></beans:value>
</util:list>
<util:list id="requiredAuths" value-type="java.lang.String">
<beans:value></beans:value>
</util:list>
</beans:beans>
+8 -1
View File
@@ -8,7 +8,10 @@ timely-balancer:
assignmentFile: /timely/balancer/assignments.csv
security:
session-max-age: 86400
allow-anonymous-access: true
cache-expiration-minutes: 60
cache-refresh-minutes: 5
allow-anonymous-http-access: true
allow-anonymous-ws-access: true
serverSsl:
certificate-file:
key-file:
@@ -24,6 +27,9 @@ timely-balancer:
trustStoreFile:
trustStoreType: jks
#trustStorePassword:
jwtCheckKeyStore:
jwtCheckKeyType:
jwtCheckKeyPassword:
cache:
enabled: false
metric-age-off-hours:
@@ -108,6 +114,7 @@ timely-balancer:
flush-interval-seconds: 30
scanner-read-ahead: 1
subscription-batch-size: 5000
incomingBufferSize: 50000000
wsClientPool:
minIdlePerKey: 0
maxIdlePerKey: 40
+291 -327
View File
@@ -1,327 +1,291 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>timely-balancer</artifactId>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
<dependency>
<groupId>com.google.flatbuffers</groupId>
<artifactId>flatbuffers-java</artifactId>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
</dependency>
<dependency>
<groupId>io.github.lukehutch</groupId>
<artifactId>fast-classpath-scanner</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</dependency>
<dependency>
<groupId>net.sourceforge.javacsv</groupId>
<artifactId>javacsv</artifactId>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-core</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-x-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<exclusions>
<exclusion>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-websocket</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-client</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-server</artifactId>
</dependency>
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
<version>${version.netty-tcnative}</version>
<classifier>${os.detected.classifier}</classifier>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-test-autoconfigure</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>assemble</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>src/main/dist/assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<goals>
<goal>copy-dependencies</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<excludeArtifactIds>junit</excludeArtifactIds>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
<execution>
<id>unpack-native</id>
<goals>
<goal>unpack</goal>
</goals>
<phase>process-test-classes</phase>
<configuration>
<artifactItems>
<artifactItem>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
<version>${version.netty-tcnative}</version>
<classifier>${os.detected.classifier}</classifier>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/native</outputDirectory>
<includes>**/*.so</includes>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-source</id>
<goals>
<goal>add-source</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<sources>
<source>src/main/generated-java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<extensions>
<!-- Use os-maven-plugin to initialize the "os.detected" properties -->
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
</extension>
</extensions>
</build>
<profiles>
<profile>
<id>native-epoll-linux</id>
<activation>
<os>
<family>!mac</family>
</os>
</activation>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<version>${version.netty}</version>
<classifier>linux-x86_64</classifier>
</dependency>
</dependencies>
</profile>
<profile>
<id>native-epoll</id>
<activation>
<os>
<family>mac</family>
</os>
</activation>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<version>${version.netty}</version>
</dependency>
</dependencies>
</profile>
</profiles>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>timely-balancer</artifactId>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
<dependency>
<groupId>com.google.flatbuffers</groupId>
<artifactId>flatbuffers-java</artifactId>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
</dependency>
<dependency>
<groupId>io.github.lukehutch</groupId>
<artifactId>fast-classpath-scanner</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</dependency>
<dependency>
<groupId>net.sourceforge.javacsv</groupId>
<artifactId>javacsv</artifactId>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-core</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-x-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<exclusions>
<exclusion>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-websocket</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-client</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-server</artifactId>
</dependency>
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-test-autoconfigure</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>assemble</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>src/main/dist/assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<goals>
<goal>copy-dependencies</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<excludeArtifactIds>junit</excludeArtifactIds>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
<execution>
<id>unpack-native</id>
<goals>
<goal>unpack</goal>
</goals>
<phase>process-test-classes</phase>
<configuration>
<artifactItems>
<artifactItem>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>${version.netty-tcnative}</version>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/native</outputDirectory>
<includes>**/*.so</includes>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-source</id>
<goals>
<goal>add-source</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<sources>
<source>src/main/generated-java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<extensions>
<!-- Use os-maven-plugin to initialize the "os.detected" properties -->
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
</extension>
</extensions>
</build>
</project>
@@ -1,7 +1,7 @@
package timely.balancer;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import com.google.common.util.concurrent.AtomicDouble;
@@ -20,38 +20,26 @@ public class ArrivalRate {
private AtomicDouble longTermRate = new AtomicDouble(0);
private AtomicLong longTermLastReset = new AtomicLong(0);
public ArrivalRate(Timer timer) {
public ArrivalRate(ScheduledExecutorService arrivalRateExecutor) {
long now = System.currentTimeMillis();
shortTermLastReset.set(now);
mediumTermLastReset.set(now);
longTermLastReset.set(now);
// 2 minute
timer.schedule(new TimerTask() {
@Override
public void run() {
resetShort();
}
}, 120000, 120000);
arrivalRateExecutor.scheduleAtFixedRate(() -> {
resetShort();
}, 2, 2, TimeUnit.MINUTES);
// 5 minute
timer.schedule(new TimerTask() {
@Override
public void run() {
resetMedium();
}
}, 300000, 300000);
arrivalRateExecutor.scheduleAtFixedRate(() -> {
resetMedium();
}, 5, 5, TimeUnit.MINUTES);
// 15 minute
timer.schedule(new TimerTask() {
@Override
public void run() {
resetLong();
}
}, 900000, 900000);
arrivalRateExecutor.scheduleAtFixedRate(() -> {
resetLong();
}, 15, 15, TimeUnit.MINUTES);
}
private void resetShort() {
@@ -25,6 +25,7 @@ import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.WriteBufferWaterMark;
import io.netty.channel.epoll.EpollChannelOption;
import io.netty.channel.epoll.EpollDatagramChannel;
import io.netty.channel.epoll.EpollEventLoopGroup;
@@ -56,6 +57,7 @@ import io.netty.handler.stream.ChunkedWriteHandler;
import io.netty.handler.timeout.IdleStateHandler;
import io.netty.util.NettyRuntime;
import io.netty.util.internal.SystemPropertyUtil;
import org.apache.accumulo.core.conf.AccumuloConfiguration;
import org.apache.curator.RetryPolicy;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
@@ -64,8 +66,11 @@ import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.data.Stat;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.context.ConfigurableApplicationContext;
import timely.auth.AuthCache;
import timely.auth.JWTTokenHandler;
import timely.balancer.configuration.BalancerConfiguration;
import timely.balancer.configuration.ClientSsl;
import timely.balancer.configuration.SpringBootstrap;
@@ -91,7 +96,7 @@ import timely.netty.tcp.TcpDecoder;
import timely.netty.tcp.TcpVersionHandler;
import timely.netty.udp.UdpDecoder;
import timely.netty.udp.UdpPacketToByteBuf;
import timely.netty.websocket.WebSocketHttpCookieHandler;
import timely.netty.websocket.WebSocketFullRequestHandler;
import timely.netty.websocket.WebSocketRequestDecoder;
import timely.netty.websocket.subscription.WSTimelyExceptionHandler;
@@ -141,8 +146,11 @@ public class Balancer {
this.balancerConfig = balancerConf;
}
protected static ConfigurableApplicationContext initializeConfiguration(String[] args) {
return new SpringApplicationBuilder(SpringBootstrap.class).web(false).run(args);
protected static ConfigurableApplicationContext initializeContext(String[] args) {
SpringApplicationBuilder builder = new SpringApplicationBuilder(SpringBootstrap.class);
builder.web(WebApplicationType.NONE);
builder.registerShutdownHook(false);
return builder.run(args);
}
private void shutdownHook() {
@@ -174,8 +182,9 @@ public class Balancer {
channelFutures.add(wsChannelHandle.close());
}
int udpChannel = 1;
for (Channel c : udpChannelHandleList) {
LOG.info("Closing udpChannelHandle");
LOG.info("Closing udpChannelHandle #" + udpChannel++);
channelFutures.add(c.close());
}
@@ -184,14 +193,13 @@ public class Balancer {
try {
f.get();
} catch (final Exception e) {
LOG.error("Error while shutting down channel: {}", f.channel().config());
LOG.error("{}", e.getMessage());
e.printStackTrace();
LOG.error("Channel:" + f.channel().config() + " -> " + e.getMessage(), e);
}
});
Integer quietPeriod = balancerConfig.getServer().getShutdownQuietPeriod();
int quietPeriod = balancerConfig.getServer().getShutdownQuietPeriod();
List<Future<?>> groupFutures = new ArrayList<>();
if (tcpBossGroup != null) {
LOG.info("Shutting down tcpBossGroup");
groupFutures.add(tcpBossGroup.shutdownGracefully(quietPeriod, 10, TimeUnit.SECONDS));
@@ -236,9 +244,7 @@ public class Balancer {
try {
f.get();
} catch (final Exception e) {
LOG.error("Error while shutting down group: {}", f.toString());
LOG.error("{}", e.getMessage());
e.printStackTrace();
LOG.error("Group:" + f.toString() + " -> " + e.getMessage(), e);
}
});
@@ -249,14 +255,29 @@ public class Balancer {
LOG.error("Error closing WebSocketRequestDecoder during shutdown", e);
}
if (this.metricResolver != null) {
try {
LOG.info("Closing metricResolver");
this.metricResolver.close();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
if (curatorFramework != null) {
try {
LOG.info("Closing curatorFramework");
curatorFramework.close();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
if (applicationContext != null) {
LOG.info("Closing applicationContext");
applicationContext.close();
}
if (curatorFramework != null) {
curatorFramework.close();
}
this.shutdown = true;
LOG.info("Server shut down.");
}
@@ -357,11 +378,15 @@ public class Balancer {
// start curator framework
RetryPolicy retryPolicy = new RetryForever(1000);
curatorFramework = CuratorFrameworkFactory.newClient(balancerConfig.getZooKeeper().getServers(), 30000, 1000,
int timeout = Long.valueOf(AccumuloConfiguration.getTimeInMillis(balancerConfig.getZooKeeper().getTimeout()))
.intValue();
curatorFramework = CuratorFrameworkFactory.newClient(balancerConfig.getZooKeeper().getServers(), timeout, 10000,
retryPolicy);
curatorFramework.start();
ensureZkPaths(curatorFramework, zkPaths);
AuthCache.configure(balancerConfig.getSecurity());
JWTTokenHandler.init(balancerConfig.getSecurity(), null);
final boolean useEpoll = useEpoll();
Class<? extends ServerSocketChannel> channelClass;
Class<? extends Channel> datagramChannelClass;
@@ -427,7 +452,8 @@ public class Balancer {
httpServer.group(httpBossGroup, httpWorkerGroup);
httpServer.channel(channelClass);
httpServer.handler(new LoggingHandler());
HttpClientPool httpClientPool = new HttpClientPool(balancerConfig, clientSSLContext);
HttpClientPool httpClientPool = new HttpClientPool(balancerConfig.getSecurity(), balancerConfig.getHttp(),
clientSSLContext);
httpServer.childHandler(setupHttpChannel(balancerConfig, sslCtx, metricResolver, httpClientPool));
httpServer.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
httpServer.option(ChannelOption.SO_BACKLOG, 128);
@@ -446,10 +472,9 @@ public class Balancer {
wsServer.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
wsServer.option(ChannelOption.SO_BACKLOG, 128);
wsServer.option(ChannelOption.SO_KEEPALIVE, true);
/* Not sure if next three lines are necessary */
/* Not sure if next two lines are necessary */
wsServer.option(ChannelOption.SO_SNDBUF, 1048576);
wsServer.option(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 838860);
wsServer.option(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 620145);
wsServer.option(ChannelOption.WRITE_BUFFER_WATER_MARK, new WriteBufferWaterMark(620145, 838860));
wsChannelHandle = wsServer.bind(wsIp, wsPort).sync().channel();
final String wsAddress = ((InetSocketAddress) wsChannelHandle.localAddress()).getAddress().getHostAddress();
@@ -474,12 +499,12 @@ public class Balancer {
public static void main(String[] args) throws Exception {
Balancer.applicationContext = Balancer.initializeConfiguration(args);
BalancerConfiguration balancerConf = applicationContext.getBean(BalancerConfiguration.class);
Balancer.applicationContext = Balancer.initializeContext(args);
BalancerConfiguration balancerConf = Balancer.applicationContext.getBean(BalancerConfiguration.class);
Balancer b = new Balancer(balancerConf);
Balancer balancer = new Balancer(balancerConf);
try {
b.run();
balancer.run();
LATCH.await();
} catch (final InterruptedException e) {
LOG.info("Server shutting down.");
@@ -487,7 +512,7 @@ public class Balancer {
LOG.error("Error running server.", e);
} finally {
try {
b.shutdown();
balancer.shutdown();
} catch (Exception e) {
System.exit(1);
}
@@ -504,19 +529,20 @@ public class Balancer {
ch.pipeline().addLast("ssl", new NonSslRedirectHandler(balancerConfig.getHttp(), sslCtx));
ch.pipeline().addLast("encoder", new HttpResponseEncoder());
ch.pipeline().addLast("decoder", new HttpRequestDecoder());
ch.pipeline().addLast("decoder", new HttpRequestDecoder(4096, 32768, 8192, true, 128));
ch.pipeline().addLast("compressor", new HttpContentCompressor());
ch.pipeline().addLast("decompressor", new HttpContentDecompressor());
ch.pipeline().addLast("aggregator", new HttpObjectAggregator(8192));
ch.pipeline().addLast("aggregator", new HttpObjectAggregator(65536));
ch.pipeline().addLast("chunker", new ChunkedWriteHandler());
ch.pipeline().addLast("queryDecoder", new timely.netty.http.HttpRequestDecoder(
balancerConfig.getSecurity(), balancerConfig.getHttp()));
ch.pipeline().addLast("fileServer", new HttpStaticFileServerHandler());
ch.pipeline().addLast("fileServer", new HttpStaticFileServerHandler().setIgnoreSslHandshakeErrors(
balancerConfig.getSecurity().getServerSsl().isUseGeneratedKeypair()));
ch.pipeline().addLast("login",
new X509LoginRequestHandler(balancerConfig.getSecurity(), balancerConfig.getHttp()));
ch.pipeline().addLast("httpRelay",
new HttpRelayHandler(balancerConfig, metricResolver, httpClientPool));
ch.pipeline().addLast("error", new TimelyExceptionHandler());
ch.pipeline().addLast("httpRelay", new HttpRelayHandler(metricResolver, httpClientPool));
ch.pipeline().addLast("error", new TimelyExceptionHandler().setIgnoreSslHandshakeErrors(
balancerConfig.getSecurity().getServerSsl().isUseGeneratedKeypair()));
}
};
}
@@ -529,7 +555,7 @@ public class Balancer {
ch.pipeline().addLast("logger", new LoggingHandler());
ch.pipeline().addLast("packetDecoder", new UdpPacketToByteBuf());
ch.pipeline().addLast("buffer", new MetricsBufferDecoder());
ch.pipeline().addLast("frame", new DelimiterBasedFrameDecoder(8192, true, Delimiters.lineDelimiter()));
ch.pipeline().addLast("frame", new DelimiterBasedFrameDecoder(65536, true, Delimiters.lineDelimiter()));
ch.pipeline().addLast("putDecoder", new UdpDecoder());
ch.pipeline().addLast("udpRelayHandler", new UdpRelayHandler(metricResolver, udpClientPool));
}
@@ -542,7 +568,7 @@ public class Balancer {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast("buffer", new MetricsBufferDecoder());
ch.pipeline().addLast("frame", new DelimiterBasedFrameDecoder(8192, true, Delimiters.lineDelimiter()));
ch.pipeline().addLast("frame", new DelimiterBasedFrameDecoder(65536, true, Delimiters.lineDelimiter()));
ch.pipeline().addLast("putDecoder", new TcpDecoder());
ch.pipeline().addLast("tcpRelayHandler", new TcpRelayHandler(metricResolver, tcpClientPools));
ch.pipeline().addLast("versionHandler", new TcpVersionHandler());
@@ -558,13 +584,13 @@ public class Balancer {
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast("ssl", sslCtx.newHandler(ch.alloc()));
ch.pipeline().addLast("httpServer", new HttpServerCodec());
ch.pipeline().addLast("aggregator", new HttpObjectAggregator(8192));
ch.pipeline().addLast("sessionExtractor", new WebSocketHttpCookieHandler(balancerConfig.getSecurity()));
ch.pipeline().addLast("aggregator", new HttpObjectAggregator(65536));
ch.pipeline().addLast("sessionExtractor", new WebSocketFullRequestHandler());
ch.pipeline().addLast("idle-handler",
new IdleStateHandler(balancerConfig.getWebsocket().getTimeout(), 0, 0));
ch.pipeline().addLast("ws-protocol",
new WebSocketServerProtocolHandler(WS_PATH, null, true, 65536, true));
new WebSocketServerProtocolHandler(WS_PATH, null, true, 65536, false, true));
ch.pipeline().addLast("wsDecoder", new WebSocketRequestDecoder(balancerConfig.getSecurity()));
ch.pipeline().addLast("httpRelay", new WsRelayHandler(balancerConfig, metricResolver, wsClientPool));
ch.pipeline().addLast("error", new WSTimelyExceptionHandler());
@@ -5,8 +5,10 @@ import javax.validation.Valid;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import org.springframework.stereotype.Component;
import org.springframework.validation.annotation.Validated;
import timely.configuration.Cache;
@Validated
@Component
@ConfigurationProperties(prefix = "timely-balancer")
public class BalancerConfiguration {
@@ -3,6 +3,7 @@ package timely.balancer.configuration;
import javax.validation.Valid;
import org.apache.commons.pool2.impl.GenericKeyedObjectPoolConfig;
import org.apache.http.impl.client.CloseableHttpClient;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import timely.configuration.Http;
@@ -10,9 +11,9 @@ public class BalancerHttp extends Http {
@Valid
@NestedConfigurationProperty
private GenericKeyedObjectPoolConfig httpClientPool = new GenericKeyedObjectPoolConfig();
private GenericKeyedObjectPoolConfig<CloseableHttpClient> httpClientPool = new GenericKeyedObjectPoolConfig<>();
public GenericKeyedObjectPoolConfig getHttpClientPool() {
public GenericKeyedObjectPoolConfig<CloseableHttpClient> getHttpClientPool() {
return httpClientPool;
}
}
@@ -4,27 +4,29 @@ import javax.validation.Valid;
import org.apache.commons.pool2.impl.GenericKeyedObjectPoolConfig;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import timely.client.tcp.TcpClient;
import timely.client.udp.UdpClient;
import timely.configuration.Server;
public class BalancerServer extends Server {
@Valid
@NestedConfigurationProperty
private GenericKeyedObjectPoolConfig udpClientPool = new GenericKeyedObjectPoolConfig();
private GenericKeyedObjectPoolConfig<UdpClient> udpClientPool = new GenericKeyedObjectPoolConfig<>();
@Valid
@NestedConfigurationProperty
private GenericKeyedObjectPoolConfig tcpClientPool = new GenericKeyedObjectPoolConfig();
private GenericKeyedObjectPoolConfig<TcpClient> tcpClientPool = new GenericKeyedObjectPoolConfig<>();
private int numTcpPools = 1;
private int tcpBufferSize = -1;
public GenericKeyedObjectPoolConfig getUdpClientPool() {
public GenericKeyedObjectPoolConfig<UdpClient> getUdpClientPool() {
return udpClientPool;
}
public GenericKeyedObjectPoolConfig getTcpClientPool() {
public GenericKeyedObjectPoolConfig<TcpClient> getTcpClientPool() {
return tcpClientPool;
}
@@ -4,15 +4,26 @@ import javax.validation.Valid;
import org.apache.commons.pool2.impl.GenericKeyedObjectPoolConfig;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import timely.client.websocket.subscription.WebSocketSubscriptionClient;
import timely.configuration.Websocket;
public class BalancerWebsocket extends Websocket {
@Valid
@NestedConfigurationProperty
private GenericKeyedObjectPoolConfig wsClientPool = new GenericKeyedObjectPoolConfig();
private GenericKeyedObjectPoolConfig<WebSocketSubscriptionClient> wsClientPool = new GenericKeyedObjectPoolConfig<>();
public GenericKeyedObjectPoolConfig getWsClientPool() {
public GenericKeyedObjectPoolConfig<WebSocketSubscriptionClient> getWsClientPool() {
return wsClientPool;
}
public int incomingBufferSize = 50000000;
public int getIncomingBufferSize() {
return incomingBufferSize;
}
public void setIncomingBufferSize(int incomingBufferSize) {
this.incomingBufferSize = incomingBufferSize;
}
}
@@ -8,6 +8,8 @@ public class ClientSsl {
private String trustStoreFile;
private String trustStoreType;
private String trustStorePassword;
private boolean useClientCert = true;
private boolean hostVerificationEnabled = true;
public String getKeyFile() {
return keyFile;
@@ -55,6 +57,21 @@ public class ClientSsl {
public void setTrustStorePassword(String trustStorePassword) {
this.trustStorePassword = trustStorePassword;
}
public void setUseClientCert(boolean useClientCert) {
this.useClientCert = useClientCert;
}
public boolean isUseClientCert() {
return useClientCert;
}
public void setHostVerificationEnabled(boolean hostVerificationEnabled) {
this.hostVerificationEnabled = hostVerificationEnabled;
}
public boolean isHostVerificationEnabled() {
return hostVerificationEnabled;
}
}
@@ -1,9 +1,14 @@
package timely.balancer.configuration;
import javax.validation.constraints.NotBlank;
public class ZooKeeper {
private String servers;
@NotBlank
private String timeout = "120s";
public String getServers() {
return servers;
}
@@ -11,4 +16,12 @@ public class ZooKeeper {
public void setServers(String servers) {
this.servers = servers;
}
public String getTimeout() {
return timeout;
}
public void setTimeout(String timeout) {
this.timeout = timeout;
}
}
@@ -1,7 +1,5 @@
package timely.balancer.connection;
import java.util.Timer;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
@@ -13,7 +11,6 @@ import timely.balancer.configuration.BalancerConfiguration;
public class TimelyBalancedHost {
private static final Logger LOG = LoggerFactory.getLogger(TimelyBalancedHost.class);
static private Timer arrivalRateTimer = new Timer("ArrivalRateTimerTBH", true);
private String host;
private int tcpPort;
@@ -24,34 +21,25 @@ public class TimelyBalancedHost {
private BalancerConfiguration balancerConfig;
private int failures = 0;
private int successes = 0;
private ArrivalRate arrivalRate = new ArrivalRate(arrivalRateTimer);
private ArrivalRate arrivalRate = null;
public TimelyBalancedHost() {
private TimelyBalancedHost() {
}
public TimelyBalancedHost(String host, int tcpPort, int httpPort, int wsPort, int udpPort) {
private TimelyBalancedHost(String host, int tcpPort, int httpPort, int wsPort, int udpPort,
ArrivalRate arrivalRate) {
this.host = host;
this.tcpPort = tcpPort;
this.httpPort = httpPort;
this.wsPort = wsPort;
this.udpPort = udpPort;
this.arrivalRate = arrivalRate;
}
public TimelyBalancedHost(String host, int basePort) {
this.host = host;
this.tcpPort = basePort;
this.httpPort = basePort + 1;
this.wsPort = basePort + 2;
this.udpPort = basePort + 3;
}
static public TimelyBalancedHost of(String host, int tcpPort, int httpPort, int wsPort, int udpPort) {
return new TimelyBalancedHost(host, tcpPort, httpPort, wsPort, udpPort);
}
static public TimelyBalancedHost of(String host, int basePort) {
return new TimelyBalancedHost(host, basePort);
static public TimelyBalancedHost of(String host, int tcpPort, int httpPort, int wsPort, int udpPort,
ArrivalRate arrivalRate) {
return new TimelyBalancedHost(host, tcpPort, httpPort, wsPort, udpPort, arrivalRate);
}
public void setBalancerConfig(BalancerConfiguration balancerConfig) {
@@ -141,11 +129,17 @@ public class TimelyBalancedHost {
}
public void arrived() {
arrivalRate.arrived();
if (this.arrivalRate != null) {
this.arrivalRate.arrived();
}
}
public double getArrivalRate() {
return arrivalRate.getRate();
if (this.arrivalRate == null) {
return 0;
} else {
return this.arrivalRate.getRate();
}
}
@Override
@@ -5,22 +5,25 @@ import javax.net.ssl.SSLContext;
import org.apache.commons.pool2.KeyedPooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.CloseableHttpClient;
import timely.balancer.configuration.BalancerSecurity;
import timely.balancer.connection.TimelyBalancedHost;
public class HttpClientFactory implements KeyedPooledObjectFactory<TimelyBalancedHost, CloseableHttpClient> {
private final SSLContext sslContext;
private final BalancerSecurity security;
public HttpClientFactory(SSLContext sslContext) {
public HttpClientFactory(BalancerSecurity security, SSLContext sslContext) {
this.security = security;
this.sslContext = sslContext;
}
@Override
public PooledObject<CloseableHttpClient> makeObject(TimelyBalancedHost k) throws Exception {
BasicCookieStore cookieJar = new BasicCookieStore();
return new DefaultPooledObject<>(timely.client.http.HttpClient.get(this.sslContext, cookieJar, false));
// disable cookie management because we are sharing connections
return new DefaultPooledObject<>(timely.client.http.HttpClient.get(this.sslContext, null,
security.getClientSsl().isHostVerificationEnabled(), security.getClientSsl().isUseClientCert()));
}
@Override
@@ -4,16 +4,13 @@ import javax.net.ssl.SSLContext;
import org.apache.commons.pool2.impl.GenericKeyedObjectPool;
import org.apache.http.impl.client.CloseableHttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import timely.balancer.configuration.BalancerConfiguration;
import timely.balancer.configuration.BalancerHttp;
import timely.balancer.configuration.BalancerSecurity;
import timely.balancer.connection.TimelyBalancedHost;
public class HttpClientPool extends GenericKeyedObjectPool<TimelyBalancedHost, CloseableHttpClient> {
private static final Logger LOG = LoggerFactory.getLogger(HttpClientPool.class);
public HttpClientPool(BalancerConfiguration config, SSLContext sslContext) {
super(new HttpClientFactory(sslContext), config.getHttp().getHttpClientPool());
public HttpClientPool(BalancerSecurity balancerSecurity, BalancerHttp http, SSLContext sslContext) {
super(new HttpClientFactory(balancerSecurity, sslContext), http.getHttpClientPool());
}
}
@@ -12,21 +12,21 @@ import timely.client.websocket.subscription.WebSocketSubscriptionClient;
public class WsClientFactory implements KeyedPooledObjectFactory<TimelyBalancedHost, WebSocketSubscriptionClient> {
private final SSLContext sslContext;
private final boolean doLogin;
private BalancerConfiguration balancerConfig;
public WsClientFactory(BalancerConfiguration balancerConfig, SSLContext sslContext) {
this.sslContext = sslContext;
this.doLogin = balancerConfig.isLoginRequired();
this.balancerConfig = balancerConfig;
this.sslContext = sslContext;
}
@Override
public PooledObject<WebSocketSubscriptionClient> makeObject(TimelyBalancedHost k) throws Exception {
int bufferSize = balancerConfig.getWebsocket().getSubscriptionBatchSize() * 500;
int bufferSize = balancerConfig.getWebsocket().getIncomingBufferSize();
WebSocketSubscriptionClient client = new WebSocketSubscriptionClient(sslContext, k.getHost(), k.getHttpPort(),
k.getWsPort(), doLogin, "", "", false, bufferSize);
k.getWsPort(), balancerConfig.getSecurity().getClientSsl().isUseClientCert(),
balancerConfig.isLoginRequired(), "", "",
balancerConfig.getSecurity().getClientSsl().isHostVerificationEnabled(), bufferSize);
return new DefaultPooledObject<>(client);
}
@@ -2,8 +2,9 @@ package timely.balancer.healthcheck;
import java.util.ArrayList;
import java.util.List;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -15,12 +16,12 @@ public class HealthChecker {
private static final Logger LOG = LoggerFactory.getLogger(HealthChecker.class);
private List<TimelyBalancedHost> timelyHosts = new ArrayList<>();
private Timer timer;
private ScheduledExecutorService executorService = Executors.newScheduledThreadPool(1);
final private TCPServerHealthCheck tcpServerHealthCheck;
private TimerTask check = new TimerTask() {
@Override
public void run() {
public HealthChecker(BalancerConfiguration config, TcpClientPool tcpClientPool) {
this.tcpServerHealthCheck = new TCPServerHealthCheck(tcpClientPool);
this.executorService.scheduleAtFixedRate(() -> {
synchronized (timelyHosts) {
for (TimelyBalancedHost h : timelyHosts) {
try {
@@ -35,13 +36,7 @@ public class HealthChecker {
}
}
}
}
};
public HealthChecker(BalancerConfiguration config, TcpClientPool tcpClientPool) {
this.tcpServerHealthCheck = new TCPServerHealthCheck(tcpClientPool);
this.timer = new Timer("HealthCheckerThread", true);
this.timer.scheduleAtFixedRate(check, 0, config.getCheckServerHealthInterval());
}, 0, config.getCheckServerHealthInterval(), TimeUnit.MILLISECONDS);
}
public void setTimelyHosts(List<TimelyBalancedHost> timelyHosts) {
@@ -50,4 +45,17 @@ public class HealthChecker {
this.timelyHosts.addAll(timelyHosts);
}
}
public void close() throws Exception {
this.executorService.shutdown();
try {
this.executorService.awaitTermination(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
} finally {
if (!this.executorService.isTerminated()) {
this.executorService.shutdownNow();
}
}
}
}
@@ -1,9 +1,8 @@
package timely.balancer.netty.http;
import static io.netty.handler.codec.http.HttpHeaders.Names.CONTENT_LENGTH;
import static io.netty.handler.codec.http.HttpHeaderNames.CONTENT_LENGTH;
import java.io.ByteArrayOutputStream;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
@@ -12,6 +11,7 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import com.google.common.collect.Multimap;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
@@ -33,26 +33,26 @@ import org.apache.http.message.BasicHeader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.StreamUtils;
import timely.api.request.AuthenticatedRequest;
import timely.api.request.MetricRequest;
import timely.api.request.timeseries.HttpRequest;
import timely.api.request.timeseries.QueryRequest;
import timely.api.response.TimelyException;
import timely.balancer.configuration.BalancerConfiguration;
import timely.auth.util.HttpHeaderUtils;
import timely.auth.util.ProxiedEntityUtils;
import timely.balancer.connection.TimelyBalancedHost;
import timely.balancer.connection.http.HttpClientPool;
import timely.balancer.resolver.MetricResolver;
import timely.netty.http.TimelyHttpHandler;
import timely.netty.http.auth.TimelyAuthenticationToken;
public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> implements TimelyHttpHandler {
private static final Logger LOG = LoggerFactory.getLogger(HttpRelayHandler.class);
private final BalancerConfiguration balancerConf;
private final HttpClientPool httpClientPool;
private MetricResolver metricResolver;
public HttpRelayHandler(BalancerConfiguration balancerConf, MetricResolver metricResolver,
HttpClientPool httpClientPool) {
this.balancerConf = balancerConf;
public HttpRelayHandler(MetricResolver metricResolver, HttpClientPool httpClientPool) {
this.metricResolver = metricResolver;
this.httpClientPool = httpClientPool;
}
@@ -67,7 +67,7 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
try {
try {
request = msg.getHttpRequest();
String originalURI = request.getUri();
String originalURI = request.uri();
originalURI = encodeURI(originalURI);
String metric = null;
@@ -86,14 +86,22 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
} else {
k = metricResolver.getHostPortKey(null);
}
k = metricResolver.getHostPortKey(metric);
client = httpClientPool.borrowObject(k);
List<Map.Entry<String, String>> headerList = request.headers().entries();
List<Header> relayedHeaderList = new ArrayList<>();
Multimap<String, String> headers = HttpHeaderUtils.toMultimap(request.headers());
if (msg instanceof AuthenticatedRequest) {
AuthenticatedRequest authenticatedRequest = (AuthenticatedRequest) msg;
TimelyAuthenticationToken token = authenticatedRequest.getToken();
if (token != null) {
if (token.getClientCert() != null) {
ProxiedEntityUtils.addProxyHeaders(headers, token.getClientCert());
}
}
}
for (Map.Entry<String, String> h : headerList) {
if (!h.getKey().equals(CONTENT_LENGTH)) {
List<Header> relayedHeaderList = new ArrayList<>();
for (Map.Entry<String, String> h : headers.entries()) {
if (!h.getKey().equals(CONTENT_LENGTH.toString())) {
relayedHeaderList.add(new BasicHeader(h.getKey(), h.getValue()));
}
}
@@ -101,14 +109,14 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
relayedHeaderList.toArray(headerArray);
String relayURI = "https://" + k.getHost() + ":" + k.getHttpPort() + originalURI;
if (request.getMethod().equals(HttpMethod.GET)) {
if (request.method().equals(HttpMethod.GET)) {
HttpUriRequest relayedRequest = new HttpGet(relayURI);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.getMethod().equals(HttpMethod.POST)) {
} else if (request.method().equals(HttpMethod.POST)) {
HttpPost relayedRequest = new HttpPost(relayURI);
String content = request.content().toString(StandardCharsets.UTF_8);
@@ -134,7 +142,7 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
e.getMessage(), e.getLocalizedMessage(), e));
return;
} finally {
if (request != null && request instanceof AutoCloseable) {
if (request != null) {
ReferenceCountUtil.release(request);
}
}
@@ -143,7 +151,6 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
this.sendHttpError(ctx, new TimelyException(HttpResponseStatus.METHOD_NOT_ALLOWED.code(),
"Method not allowed", "Method not allowed"));
} else {
// buf = JsonUtil.getObjectMapper().writeValueAsBytes(msg);
ByteArrayOutputStream baos = new MyByteArrayOutputStream();
StreamUtils.copy(relayedResponse.getEntity().getContent(), baos);
@@ -153,8 +160,6 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
for (Header h : relayedResponse.getAllHeaders()) {
response.headers().add(h.getName(), h.getValue());
}
// response.headers().set(HttpHeaders.Names.CONTENT_TYPE,
// Constants.JSON_TYPE);
response.headers().set(CONTENT_LENGTH, response.content().readableBytes());
sendResponse(ctx, response);
}
@@ -165,7 +170,7 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
if (client != null && k != null) {
httpClientPool.returnObject(k, client);
} else {
LOG.error("NOT RETURNING CONNECTION! " + msg.getHttpRequest().getUri());
LOG.error("NOT RETURNING CONNECTION! " + msg.getHttpRequest().uri());
}
}
}
@@ -197,13 +202,4 @@ public class HttpRelayHandler extends SimpleChannelInboundHandler<HttpRequest> i
}
return s;
}
public static void main(String[] args) {
String s = "https://127.0.0.1:4243/api/query?start=1356998400000&end=1528398304000&m=sum:rate{false,100,0}:sys.thermal.gauge{class=aws}";
URI uri = URI.create(encodeURI(s));
System.out.println(uri.toString());
}
}
@@ -1,11 +1,16 @@
package timely.balancer.netty.ws;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import javax.websocket.CloseReason;
import javax.websocket.EndpointConfig;
import javax.websocket.Session;
import com.google.common.collect.HashMultimap;
import com.google.common.collect.Multimap;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.HttpResponseStatus;
import io.netty.handler.codec.http.websocketx.PingWebSocketFrame;
@@ -14,22 +19,45 @@ import io.netty.util.concurrent.ScheduledFuture;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import timely.api.response.TimelyException;
import timely.auth.util.ProxiedEntityUtils;
import timely.client.websocket.ClientHandler;
import timely.netty.http.auth.TimelyAuthenticationToken;
public class WsClientHandler extends ClientHandler {
private static final Logger LOG = LoggerFactory.getLogger(WsClientHandler.class);
private final ChannelHandlerContext ctx;
private final TimelyAuthenticationToken token;
private ScheduledFuture<?> ping;
public WsClientHandler(ChannelHandlerContext ctx, int pingRate) {
public WsClientHandler(ChannelHandlerContext ctx, TimelyAuthenticationToken token, int pingRate) {
this.ctx = ctx;
this.token = token;
this.ping = this.ctx.executor().scheduleAtFixedRate(() -> {
LOG.trace("Sending ping on channel {}", ctx.channel());
ctx.writeAndFlush(new PingWebSocketFrame());
}, pingRate, pingRate, TimeUnit.SECONDS);
}
@Override
public void beforeRequest(Map<String, List<String>> headers) {
if (token.getClientCert() != null) {
Multimap<String, String> proxyRequestHeaders = HashMultimap.create();
ProxiedEntityUtils.addProxyHeaders(proxyRequestHeaders, token.getClientCert());
for (String s : proxyRequestHeaders.keySet()) {
headers.put(s, new ArrayList<>(proxyRequestHeaders.get(s)));
}
}
Multimap<String, String> originalRequestHeaders = token.getHttpHeaders();
for (String s : originalRequestHeaders.keySet()) {
if (!headers.containsKey(s)) {
// add pre-existing values if key does not exist in proxyRequest
headers.put(s, new ArrayList<>(originalRequestHeaders.get(s)));
}
}
}
@Override
public void onOpen(Session session, EndpointConfig config) {
session.addMessageHandler(String.class, message -> {
@@ -9,6 +9,7 @@ import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module;
import com.google.common.collect.Multimap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;
@@ -24,6 +25,7 @@ import timely.api.request.subscription.CreateSubscription;
import timely.api.request.subscription.RemoveSubscription;
import timely.api.request.subscription.SubscriptionRequest;
import timely.api.response.TimelyException;
import timely.auth.util.ProxiedEntityUtils;
import timely.balancer.configuration.BalancerConfiguration;
import timely.balancer.connection.TimelyBalancedHost;
import timely.balancer.connection.ws.WsClientPool;
@@ -64,6 +66,10 @@ public class WsRelayHandler extends SimpleChannelInboundHandler<SubscriptionRequ
String subscriptionId;
try {
String metric = null;
Multimap<String, String> headers = msg.getRequestHeaders();
ProxiedEntityUtils.addProxyHeaders(headers, msg.getToken().getClientCert());
if (msg instanceof CreateSubscription) {
CreateSubscription create = (CreateSubscription) msg;
final String currentSubscriptionId = create.getSubscriptionId();
@@ -82,7 +88,7 @@ public class WsRelayHandler extends SimpleChannelInboundHandler<SubscriptionRequ
if (metricToClientMap != null) {
synchronized (metricToClientMap) {
// close all clients for this subscriptionId
LOG.info("Channel closed, closing subscriptions for subscriptionId:%s",
LOG.debug("Channel closed, closing subscriptions for subscriptionId:{}",
currentSubscriptionId);
for (Map.Entry<String, WsClientHolder> entry : metricToClientMap.entrySet()) {
entry.getValue().close(wsClientPool);
@@ -102,7 +108,7 @@ public class WsRelayHandler extends SimpleChannelInboundHandler<SubscriptionRequ
if (origContext == null) {
// send error because user never sent a CreateSubscription
// and we have nowhere to send the results
LOG.info("ChannelHandlerContext not found for subscriptionId:%s - createSubscription not called?",
LOG.info("ChannelHandlerContext not found for subscriptionId:{} - createSubscription not called?",
subscriptionId);
sendErrorResponse(ctx, HttpResponseStatus.BAD_REQUEST,
new IllegalArgumentException("Must call create first"));
@@ -118,7 +124,8 @@ public class WsRelayHandler extends SimpleChannelInboundHandler<SubscriptionRequ
if (hostClientHolder == null) {
k = metricResolver.getHostPortKey(metric);
client = wsClientPool.borrowObject(k);
client.open(new WsClientHandler(origContext, (balancerConfig.getWebsocket().getTimeout() / 2)));
client.open(new WsClientHandler(origContext, add.getToken(),
(balancerConfig.getWebsocket().getTimeout() / 2)));
metricToClientMap.put(metric, new WsClientHolder(k, client));
} else {
client = hostClientHolder.getClient();
@@ -155,7 +162,7 @@ public class WsRelayHandler extends SimpleChannelInboundHandler<SubscriptionRequ
if (hostClientHolder != null) {
hostClientHolder.getClient().removeSubscription(metric);
} else {
LOG.info("client not found for subscriptionId:%s metric:%s", subscriptionId, metric);
LOG.info("client not found for subscriptionId:{} metric:{}", subscriptionId, metric);
}
}
}
@@ -7,6 +7,7 @@ import static timely.balancer.Balancer.LEADER_LATCH_PATH;
import static timely.store.cache.DataStoreCache.NON_CACHED_METRICS;
import static timely.store.cache.DataStoreCache.NON_CACHED_METRICS_LOCK_PATH;
import java.io.FileNotFoundException;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.Collection;
@@ -19,10 +20,10 @@ import java.util.List;
import java.util.Map;
import java.util.Random;
import java.util.Set;
import java.util.Timer;
import java.util.TimerTask;
import java.util.TreeMap;
import java.util.TreeSet;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
@@ -71,8 +72,8 @@ public class BalancedMetricResolver implements MetricResolver {
private List<TimelyBalancedHost> serverList = new ArrayList<>();
private Random r = new Random();
final private HealthChecker healthChecker;
private Timer timer = new Timer("RebalanceTimer", true);
private Timer arrivalRateTimer = new Timer("ArrivalRateTimerResolver", true);
private ScheduledExecutorService assignmentExecutor = Executors.newScheduledThreadPool(2);
private ScheduledExecutorService arrivalRateExecutor = Executors.newScheduledThreadPool(2);
private int roundRobinCounter = 0;
private Set<String> nonCachedMetrics = new HashSet<>();
private DistributedAtomicValue nonCachedMetricsIP;
@@ -105,25 +106,15 @@ public class BalancedMetricResolver implements MetricResolver {
@Override
public void childEvent(CuratorFramework curatorFramework, TreeCacheEvent event) throws Exception {
LOG.info("Handling assignments event {}. assignmentsLastUpdatedInHdfs:{}", event.getType().toString(),
LOG.debug("Handling assignments event {}. assignmentsLastUpdatedInHdfs:{}", event.getType().toString(),
new Date(assignmentsLastUpdatedInHdfs.get().postValue()));
if (event.getType().equals(TreeCacheEvent.Type.NODE_UPDATED)) {
long lastLocalUpdate = assignmentsLastUpdatedLocal.get();
long lastHdfsUpdate = assignmentsLastUpdatedInHdfs.get().postValue();
if (lastHdfsUpdate > lastLocalUpdate) {
LOG.info("Reading assignments from hdfs lastHdfsUpdate ({}) > lastLocalUpdate ({})",
new Date(lastHdfsUpdate), new Date(lastLocalUpdate));
readAssignmentsFromHdfs();
} else {
LOG.info("Not reading assignments from hdfs lastHdfsUpdate ({}) <= lastLocalUpdate ({})",
new Date(lastHdfsUpdate), new Date(lastLocalUpdate));
}
readAssignmentsFromHdfs(true);
}
}
};
try {
TreeCache assignmentTreeCache = new TreeCache(curatorFramework, ASSIGNMENTS_LAST_UPDATED_PATH);
try (TreeCache assignmentTreeCache = new TreeCache(curatorFramework, ASSIGNMENTS_LAST_UPDATED_PATH)) {
assignmentTreeCache.getListenable().addListener(assignmentListener);
assignmentTreeCache.start();
} catch (Exception e) {
@@ -138,14 +129,13 @@ public class BalancedMetricResolver implements MetricResolver {
@Override
public void childEvent(CuratorFramework curatorFramework, TreeCacheEvent event) throws Exception {
if (event.getType().equals(TreeCacheEvent.Type.NODE_UPDATED)) {
LOG.info("Handling nonCachedMetricsIP event {}", event.getType().toString());
LOG.debug("Handling nonCachedMetricsIP event {}", event.getType().toString());
readNonCachedMetricsIP();
}
}
};
try {
TreeCache nonCachedMetricsTreeCache = new TreeCache(curatorFramework, NON_CACHED_METRICS);
try (TreeCache nonCachedMetricsTreeCache = new TreeCache(curatorFramework, NON_CACHED_METRICS)) {
nonCachedMetricsTreeCache.getListenable().addListener(nonCachedMetricsListener);
nonCachedMetricsTreeCache.start();
} catch (Exception e) {
@@ -160,55 +150,55 @@ public class BalancedMetricResolver implements MetricResolver {
}
readNonCachedMetricsIP();
readAssignmentsFromHdfs();
readAssignmentsFromHdfs(false);
timer.schedule(new TimerTask() {
@Override
public void run() {
if (isLeader.get()) {
try {
balance();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
}
}, 900000, 900000);
timer.schedule(new TimerTask() {
@Override
public void run() {
assignmentExecutor.scheduleAtFixedRate(() -> {
if (isLeader.get()) {
try {
if (isLeader.get()) {
long lastLocalUpdate = assignmentsLastUpdatedLocal.get();
long lastHdfsUpdate = assignmentsLastUpdatedInHdfs.get().postValue();
if (lastLocalUpdate > lastHdfsUpdate) {
LOG.info("Writing assignments to hdfs lastLocalUpdate ({}) > lastHdfsUpdate ({})",
new Date(lastLocalUpdate), new Date(lastHdfsUpdate));
writeAssigmentsToHdfs();
}
}
balance();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
}, 10000, 60000);
}, 15, 15, TimeUnit.MINUTES);
assignmentExecutor.scheduleAtFixedRate(() -> {
try {
if (isLeader.get()) {
long lastLocalUpdate = assignmentsLastUpdatedLocal.get();
long lastHdfsUpdate = assignmentsLastUpdatedInHdfs.get().postValue();
if (lastLocalUpdate > lastHdfsUpdate) {
LOG.debug("Leader writing assignments to hdfs lastLocalUpdate ({}) > lastHdfsUpdate ({})",
new Date(lastLocalUpdate), new Date(lastHdfsUpdate));
writeAssignmentsToHdfs();
} else {
LOG.trace("Leader not writing assignments to hdfs lastLocalUpdate ({}) <= lastHdfsUpdate ({})",
new Date(lastLocalUpdate), new Date(lastHdfsUpdate));
}
} else {
readAssignmentsFromHdfs(true);
}
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}, 10, 60, TimeUnit.SECONDS);
}
private void readNonCachedMetricsIP() {
Set<String> currentNonCachedMetricsDistributed = new TreeSet<>();
try {
nonCachedMetricsIPRWLock.readLock().acquire();
boolean acquired = false;
while (!acquired) {
acquired = nonCachedMetricsIPRWLock.readLock().acquire(60, TimeUnit.SECONDS);
}
byte[] currentNonCachedMetricsDistributedBytes = nonCachedMetricsIP.get().postValue();
if (currentNonCachedMetricsDistributedBytes != null) {
try {
currentNonCachedMetricsDistributed = SerializationUtils
.deserialize(currentNonCachedMetricsDistributedBytes);
} catch (Exception e) {
LOG.error(e.getMessage());
LOG.error(e.getMessage(), e);
currentNonCachedMetricsDistributed = new TreeSet<>();
}
}
@@ -225,10 +215,10 @@ public class BalancedMetricResolver implements MetricResolver {
nonCachedMetricsLocalLock.writeLock().lock();
try {
if (nonCachedMetrics.containsAll(currentNonCachedMetricsDistributed)) {
LOG.info("local nonCachedMetrics already contains {}", currentNonCachedMetricsDistributed);
LOG.debug("local nonCachedMetrics already contains {}", currentNonCachedMetricsDistributed);
} else {
currentNonCachedMetricsDistributed.removeAll(nonCachedMetrics);
LOG.info("Adding {} to local nonCachedMetrics", currentNonCachedMetricsDistributed);
LOG.debug("Adding {} to local nonCachedMetrics", currentNonCachedMetricsDistributed);
nonCachedMetrics.addAll(currentNonCachedMetricsDistributed);
}
} catch (Exception e) {
@@ -249,7 +239,7 @@ public class BalancedMetricResolver implements MetricResolver {
public void isLeader() {
LOG.info("this balancer is the leader");
isLeader.set(true);
writeAssigmentsToHdfs();
writeAssignmentsToHdfs();
}
@Override
@@ -265,17 +255,17 @@ public class BalancedMetricResolver implements MetricResolver {
private void startServiceListener(CuratorFramework curatorFramework) {
try {
ServiceDiscovery discovery = ServiceDiscoveryBuilder.builder(ServerDetails.class).client(curatorFramework)
.basePath(SERVICE_DISCOVERY_PATH).build();
ServiceDiscovery<ServerDetails> discovery = ServiceDiscoveryBuilder.builder(ServerDetails.class)
.client(curatorFramework).basePath(SERVICE_DISCOVERY_PATH).build();
discovery.start();
Collection<ServiceInstance> instances = discovery.queryForInstances("timely-server");
Collection<ServiceInstance<ServerDetails>> instances = discovery.queryForInstances("timely-server");
balancerLock.writeLock().lock();
try {
for (ServiceInstance si : instances) {
ServerDetails pl = (ServerDetails) si.getPayload();
for (ServiceInstance<ServerDetails> si : instances) {
ServerDetails pl = si.getPayload();
TimelyBalancedHost tbh = TimelyBalancedHost.of(pl.getHost(), pl.getTcpPort(), pl.getHttpPort(),
pl.getWsPort(), pl.getUdpPort());
pl.getWsPort(), pl.getUdpPort(), new ArrivalRate(arrivalRateExecutor));
LOG.info("adding service {} host:{} tcpPort:{} httpPort:{} wsPort:{} udpPort:{}", si.getId(),
pl.getHost(), pl.getTcpPort(), pl.getHttpPort(), pl.getWsPort(), pl.getUdpPort());
tbh.setBalancerConfig(balancerConfig);
@@ -286,7 +276,8 @@ public class BalancedMetricResolver implements MetricResolver {
balancerLock.writeLock().unlock();
}
final ServiceCache serviceCache = discovery.serviceCacheBuilder().name("timely-server").build();
final ServiceCache<ServerDetails> serviceCache = discovery.serviceCacheBuilder().name("timely-server")
.build();
ServiceCacheListener listener = new ServiceCacheListener() {
@Override
@@ -294,12 +285,13 @@ public class BalancedMetricResolver implements MetricResolver {
boolean rebalanceNeeded = false;
balancerLock.writeLock().lock();
try {
List<ServiceInstance> instances = serviceCache.getInstances();
List<ServiceInstance<ServerDetails>> instances = serviceCache.getInstances();
Set<TimelyBalancedHost> availableHosts = new HashSet<>();
for (ServiceInstance si : instances) {
for (ServiceInstance<ServerDetails> si : instances) {
ServerDetails pl = (ServerDetails) si.getPayload();
TimelyBalancedHost tbh = TimelyBalancedHost.of(pl.getHost(), pl.getTcpPort(),
pl.getHttpPort(), pl.getWsPort(), pl.getUdpPort());
pl.getHttpPort(), pl.getWsPort(), pl.getUdpPort(),
new ArrivalRate(arrivalRateExecutor));
tbh.setBalancerConfig(balancerConfig);
availableHosts.add(tbh);
}
@@ -341,14 +333,16 @@ public class BalancedMetricResolver implements MetricResolver {
assignMetric(s, h);
LOG.debug("Assigned server removed. Assigning {} to server {}:{}", s, h.getTcpPort());
}
writeAssigmentsToHdfs();
}
} finally {
balancerLock.writeLock().unlock();
}
if (isLeader.get() && rebalanceNeeded) {
balance();
if (isLeader.get()) {
if (rebalanceNeeded) {
balance();
}
writeAssignmentsToHdfs();
}
}
@@ -505,21 +499,21 @@ public class BalancedMetricResolver implements MetricResolver {
}
if (highHosts.isEmpty() && lowHosts.isEmpty()) {
LOG.info("Host's arrival rates are within {} of the average", controlBandPercentage);
LOG.debug("Host's arrival rates are within {} of the average", controlBandPercentage);
} else if (!highHosts.isEmpty() && !lowHosts.isEmpty()) {
LOG.info("begin rebalancing {}", BalanceType.HIGH_LOW);
LOG.debug("begin rebalancing {}", BalanceType.HIGH_LOW);
numReassigned = rebalance(highHosts, lowHosts, calculatedRates, averageArrivalRate,
BalanceType.HIGH_LOW);
LOG.info("end rebalancing {} - reassigned {}", BalanceType.HIGH_LOW, numReassigned);
LOG.debug("end rebalancing {} - reassigned {}", BalanceType.HIGH_LOW, numReassigned);
} else if (lowHosts.isEmpty()) {
LOG.info("begin rebalancing {}", BalanceType.HIGH_AVG);
LOG.debug("begin rebalancing {}", BalanceType.HIGH_AVG);
numReassigned = rebalance(highHosts, avgHosts, calculatedRates, averageArrivalRate,
BalanceType.HIGH_AVG);
LOG.info("end rebalancing {} - reassigned {}", BalanceType.HIGH_AVG, numReassigned);
LOG.debug("end rebalancing {} - reassigned {}", BalanceType.HIGH_AVG, numReassigned);
} else {
LOG.info("begin rebalancing {}", BalanceType.AVG_LOW);
LOG.debug("begin rebalancing {}", BalanceType.AVG_LOW);
numReassigned = rebalance(avgHosts, lowHosts, calculatedRates, averageArrivalRate, BalanceType.AVG_LOW);
LOG.info("end rebalancing {} - reassigned {}", BalanceType.AVG_LOW, numReassigned);
LOG.debug("end rebalancing {} - reassigned {}", BalanceType.AVG_LOW, numReassigned);
}
}
return numReassigned;
@@ -556,7 +550,8 @@ public class BalancedMetricResolver implements MetricResolver {
// sort metrics by rate
Map<Double, String> rateSortedMetrics = new TreeMap<>(Collections.reverseOrder());
for (Map.Entry<String, ArrivalRate> e : tempMetricMap.entrySet()) {
if (metricToHostMap.get(e.getKey()).equals(h)) {
TimelyBalancedHost tbh = metricToHostMap.get(e.getKey());
if (tbh != null && tbh.equals(h)) {
rateSortedMetrics.put(e.getValue().getRate(), e.getKey());
}
}
@@ -633,7 +628,7 @@ public class BalancedMetricResolver implements MetricResolver {
private void addNonCachedMetrics(Collection<String> nonCachedMetricsUpdate) {
if (!nonCachedMetricsUpdate.isEmpty()) {
try {
LOG.info("Adding {} to local nonCachedMetrics", nonCachedMetricsUpdate);
LOG.debug("Adding {} to local nonCachedMetrics", nonCachedMetricsUpdate);
nonCachedMetricsLocalLock.writeLock().lock();
try {
nonCachedMetrics.addAll(nonCachedMetricsUpdate);
@@ -642,7 +637,10 @@ public class BalancedMetricResolver implements MetricResolver {
}
try {
nonCachedMetricsIPRWLock.writeLock().acquire();
boolean acquired = false;
while (!acquired) {
acquired = nonCachedMetricsIPRWLock.writeLock().acquire(60, TimeUnit.SECONDS);
}
byte[] currentNonCachedMetricsDistributedBytes = nonCachedMetricsIP.get().postValue();
Set<String> currentNonCachedMetricsIP;
if (currentNonCachedMetricsDistributedBytes == null) {
@@ -652,10 +650,10 @@ public class BalancedMetricResolver implements MetricResolver {
.deserialize(currentNonCachedMetricsDistributedBytes);
}
if (currentNonCachedMetricsIP.containsAll(nonCachedMetricsUpdate)) {
LOG.info("nonCachedMetricsIP already contains {}", nonCachedMetricsUpdate);
LOG.debug("nonCachedMetricsIP already contains {}", nonCachedMetricsUpdate);
} else {
nonCachedMetricsUpdate.removeAll(currentNonCachedMetricsIP);
LOG.info("Adding {} to nonCachedMetricsIP", nonCachedMetricsUpdate);
LOG.debug("Adding {} to nonCachedMetricsIP", nonCachedMetricsUpdate);
TreeSet<String> updateSet = new TreeSet<>();
updateSet.addAll(currentNonCachedMetricsIP);
updateSet.addAll(nonCachedMetricsUpdate);
@@ -689,7 +687,7 @@ public class BalancedMetricResolver implements MetricResolver {
try {
lock.writeLock().release();
} catch (Exception e3) {
LOG.error(e3.getMessage());
LOG.error(e3.getMessage(), e3);
}
}
}
@@ -720,7 +718,7 @@ public class BalancedMetricResolver implements MetricResolver {
try {
for (String r : nonCachedMetrics) {
if (metricName.matches(r)) {
LOG.info("Adding {} to list of non-cached metrics", metricName);
LOG.debug("Adding {} to list of non-cached metrics", metricName);
addNonCachedMetrics(Collections.singleton(metricName));
return false;
}
@@ -740,18 +738,26 @@ public class BalancedMetricResolver implements MetricResolver {
boolean chooseMetricSpecificHost;
try {
chooseMetricSpecificHost = shouldCache(metric) ? true : false;
ArrivalRate rate = metricMap.get(metric);
if ((chooseMetricSpecificHost && rate == null) || (!chooseMetricSpecificHost && rate != null)) {
if (!balancerLock.isWriteLockedByCurrentThread()) {
balancerLock.readLock().unlock();
balancerLock.writeLock().lock();
}
}
if (chooseMetricSpecificHost) {
ArrivalRate rate = metricMap.get(metric);
if (rate == null) {
rate = new ArrivalRate(arrivalRateTimer);
if (!balancerLock.isWriteLockedByCurrentThread()) {
balancerLock.readLock().unlock();
balancerLock.writeLock().lock();
}
rate = new ArrivalRate(arrivalRateExecutor);
// already write locked above
metricMap.put(metric, rate);
}
rate.arrived();
} else {
if (rate != null) {
// already write locked above
metricMap.remove(metric);
}
metric = null;
}
@@ -769,7 +775,7 @@ public class BalancedMetricResolver implements MetricResolver {
} else if (!tbh.isUp()) {
TimelyBalancedHost oldTbh = tbh;
tbh = getLeastUsedHost();
LOG.info("rebalancing from host that is down: reassigning metric {} from server {}:{} to {}:{}",
LOG.debug("rebalancing from host that is down: reassigning metric {} from server {}:{} to {}:{}",
metric, oldTbh.getHost(), oldTbh.getTcpPort(), tbh.getHost(), tbh.getTcpPort());
if (!balancerLock.isWriteLockedByCurrentThread()) {
balancerLock.readLock().unlock();
@@ -874,11 +880,44 @@ public class BalancedMetricResolver implements MetricResolver {
return tbh;
}
private void readAssignmentsFromHdfs() {
private void createAssignmentFile() {
try {
LOG.info("Creating assignment file: " + balancerConfig.getAssignmentFile());
Configuration configuration = new Configuration();
FileSystem fs = FileSystem.get(configuration);
Path assignmentFile = new Path(balancerConfig.getAssignmentFile());
fs.create(assignmentFile);
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
private void readAssignmentsFromHdfs(boolean checkIfNecessary) {
try {
long lastLocalUpdate = assignmentsLastUpdatedLocal.get();
long lastHdfsUpdate = assignmentsLastUpdatedInHdfs.get().postValue();
if (checkIfNecessary) {
if (lastHdfsUpdate <= lastLocalUpdate) {
LOG.debug("Not reading assignments from hdfs lastHdfsUpdate ({}) <= lastLocalUpdate ({})",
new Date(lastHdfsUpdate), new Date(lastLocalUpdate));
return;
}
}
LOG.debug("Reading assignments from hdfs lastHdfsUpdate ({}) > lastLocalUpdate ({})",
new Date(lastHdfsUpdate), new Date(lastLocalUpdate));
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
// proceed with reading from HDFS
Map<String, TimelyBalancedHost> assignedMetricToHostMap = new TreeMap<>();
try {
assignmentsIPRWLock.readLock().acquire();
boolean acquired = false;
while (!acquired) {
acquired = assignmentsIPRWLock.readLock().acquire(60, TimeUnit.SECONDS);
}
balancerLock.writeLock().lock();
Configuration configuration = new Configuration();
FileSystem fs = FileSystem.get(configuration);
@@ -911,8 +950,14 @@ public class BalancedMetricResolver implements MetricResolver {
metricToHostMap.clear();
for (Map.Entry<String, TimelyBalancedHost> e : assignedMetricToHostMap.entrySet()) {
if (shouldCache(e.getKey())) {
metricToHostMap.put(e.getKey(), e.getValue());
if (StringUtils.isNotBlank(e.getKey()) && e.getValue() != null) {
if (shouldCache(e.getKey())) {
metricToHostMap.put(e.getKey(), e.getValue());
}
} else {
Exception e1 = new IllegalStateException(
"Bad assignment metric:" + e.getKey() + " host:" + e.getValue());
LOG.warn(e1.getMessage(), e1);
}
}
@@ -928,9 +973,15 @@ public class BalancedMetricResolver implements MetricResolver {
} finally {
nonCachedMetricsLocalLock.readLock().unlock();
}
// remove metric from metricMap (ArrivalRate) if not being cached
metricMap.entrySet().removeIf(e -> !metricToHostMap.containsKey(e.getKey()));
assignmentsLastUpdatedLocal.set(assignmentsLastUpdatedInHdfs.get().postValue());
LOG.info("Read {} assignments from hdfs lastHdfsUpdate = lastLocalUpdate ({})", metricToHostMap.size(),
new Date(assignmentsLastUpdatedLocal.get()));
} catch (FileNotFoundException e) {
createAssignmentFile();
metricToHostMap.clear();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
} finally {
@@ -943,11 +994,14 @@ public class BalancedMetricResolver implements MetricResolver {
}
}
private void writeAssigmentsToHdfs() {
private void writeAssignmentsToHdfs() {
CsvWriter writer = null;
try {
assignmentsIPRWLock.writeLock().acquire();
boolean acquired = false;
while (!acquired) {
acquired = assignmentsIPRWLock.writeLock().acquire(60, TimeUnit.SECONDS);
}
balancerLock.readLock().lock();
nonCachedMetricsLocalLock.readLock().lock();
try {
@@ -982,8 +1036,10 @@ public class BalancedMetricResolver implements MetricResolver {
if (!assignmentsLastUpdatedInHdfs.get().succeeded()) {
assignmentsLastUpdatedInHdfs.forceSet(now);
}
LOG.info("Wrote {} assignments to hdfs lastHdfsUpdate = lastLocalUpdate ({})",
LOG.debug("Wrote {} assignments to hdfs lastHdfsUpdate = lastLocalUpdate ({})",
metricToHostMap.size(), new Date(assignmentsLastUpdatedLocal.get()));
// remove metric from metricMap (ArrivalRate) if not being cached
metricMap.entrySet().removeIf(e -> !metricToHostMap.containsKey(e.getKey()));
}
} finally {
nonCachedMetricsLocalLock.readLock().unlock();
@@ -1005,13 +1061,50 @@ public class BalancedMetricResolver implements MetricResolver {
private void assignMetric(String metric, TimelyBalancedHost tbh) {
if (isLeader.get()) {
balancerLock.writeLock().lock();
try {
metricToHostMap.put(metric, tbh);
assignmentsLastUpdatedLocal.set(System.currentTimeMillis());
} finally {
balancerLock.writeLock().unlock();
if (StringUtils.isNotBlank(metric) && tbh != null) {
balancerLock.writeLock().lock();
try {
metricToHostMap.put(metric, tbh);
assignmentsLastUpdatedLocal.set(System.currentTimeMillis());
} finally {
balancerLock.writeLock().unlock();
}
} else {
Exception e = new IllegalStateException("Bad assignment metric:" + metric + " host:" + tbh);
LOG.warn(e.getMessage(), e);
}
}
}
@Override
public void close() throws Exception {
this.assignmentExecutor.shutdown();
try {
this.assignmentExecutor.awaitTermination(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
} finally {
if (!this.assignmentExecutor.isTerminated()) {
this.assignmentExecutor.shutdownNow();
}
}
this.arrivalRateExecutor.shutdown();
try {
this.arrivalRateExecutor.awaitTermination(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
} finally {
if (!this.arrivalRateExecutor.isTerminated()) {
this.arrivalRateExecutor.shutdownNow();
}
}
try {
this.healthChecker.close();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
}
}
@@ -7,4 +7,6 @@ public interface MetricResolver {
public TimelyBalancedHost getHostPortKeyIngest(String metric);
public TimelyBalancedHost getHostPortKey(String metric);
public void close() throws Exception;
}
+150 -149
View File
@@ -1,149 +1,150 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>timely-client</artifactId>
<properties>
<!-- PEP 440 compliant version gets substituted into src/main/python/template/setup.py -->
<python.client.version>1.1.1.dev0</python.client.version>
</properties>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-client</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-container-grizzly-client</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>filter-setup</id>
<goals>
<goal>copy-resources</goal>
</goals>
<phase>process-resources</phase>
<configuration>
<outputDirectory>${project.basedir}/src/main/python</outputDirectory>
<resources>
<resource>
<directory>${project.basedir}/src/main/python/template</directory>
<includes>
<include>setup.py</include>
</includes>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>python</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>python-dist</id>
<goals>
<goal>exec</goal>
</goals>
<phase>package</phase>
<configuration>
<executable>python</executable>
<workingDirectory>${project.basedir}/src/main/python</workingDirectory>
<commandlineArgs>setup.py sdist --dist-dir ${project.build.directory}</commandlineArgs>
</configuration>
</execution>
<execution>
<id>python-clean</id>
<goals>
<goal>exec</goal>
</goals>
<phase>package</phase>
<configuration>
<executable>rm</executable>
<workingDirectory>${project.basedir}/src/main/python</workingDirectory>
<commandlineArgs>-rf timely.egg-info setup.py</commandlineArgs>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>timely-client</artifactId>
<properties>
<!-- PEP 440 compliant version gets substituted into src/main/python/template/setup.py -->
<!-- use .dev0 instead of -SNAPSHOT -->
<python.client.version>1.3.3.dev0</python.client.version>
</properties>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-client</artifactId>
</dependency>
<dependency>
<groupId>org.glassfish.tyrus</groupId>
<artifactId>tyrus-container-grizzly-client</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>filter-setup</id>
<goals>
<goal>copy-resources</goal>
</goals>
<phase>process-resources</phase>
<configuration>
<outputDirectory>${project.basedir}/src/main/python</outputDirectory>
<resources>
<resource>
<directory>${project.basedir}/src/main/python/template</directory>
<includes>
<include>setup.py</include>
</includes>
<filtering>true</filtering>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>python</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>python-dist</id>
<goals>
<goal>exec</goal>
</goals>
<phase>package</phase>
<configuration>
<executable>python</executable>
<workingDirectory>${project.basedir}/src/main/python</workingDirectory>
<commandlineArgs>setup.py sdist --dist-dir ${project.build.directory}</commandlineArgs>
</configuration>
</execution>
<execution>
<id>python-clean</id>
<goals>
<goal>exec</goal>
</goals>
<phase>package</phase>
<configuration>
<executable>rm</executable>
<workingDirectory>${project.basedir}/src/main/python</workingDirectory>
<commandlineArgs>-rf timely.egg-info setup.py</commandlineArgs>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
@@ -2,12 +2,14 @@ package timely.api.response;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.apache.commons.lang3.builder.ToStringBuilder;
import timely.model.Metric;
import timely.model.Tag;
@JsonPropertyOrder({ "metric", "timestamp", "value", "tags", "subscriptionId", "complete" })
public class MetricResponse {
private String metric;
@@ -1,12 +1,13 @@
package timely.client.http;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import org.apache.http.client.CookieStore;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.DefaultHostnameVerifier;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
@@ -40,14 +41,31 @@ public class HttpClient {
}
public static CloseableHttpClient get(SSLContext ssl, CookieStore cookieStore, boolean hostVerificationEnabled) {
RequestConfig defaultRequestConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build();
return get(ssl, cookieStore, hostVerificationEnabled, false);
}
HttpClientBuilder builder = HttpClients.custom().setSSLContext(ssl).setDefaultCookieStore(cookieStore)
.setDefaultRequestConfig(defaultRequestConfig);
public static CloseableHttpClient get(SSLContext ssl, CookieStore cookieStore, boolean hostVerificationEnabled,
boolean clientAuth) {
HostnameVerifier hostnameVerifier;
if (hostVerificationEnabled) {
builder.setSSLHostnameVerifier(new DefaultHostnameVerifier());
hostnameVerifier = SSLConnectionSocketFactory.getDefaultHostnameVerifier();
} else {
builder.setSSLHostnameVerifier(new NoopHostnameVerifier());
hostnameVerifier = new NoopHostnameVerifier();
}
RequestConfig defaultRequestConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build();
HttpClientBuilder builder;
if (cookieStore == null) {
builder = HttpClients.custom().setSSLContext(ssl).disableCookieManagement()
.setDefaultRequestConfig(defaultRequestConfig).setSSLHostnameVerifier(hostnameVerifier);
} else {
builder = HttpClients.custom().setSSLContext(ssl).setDefaultCookieStore(cookieStore)
.setDefaultRequestConfig(defaultRequestConfig).setSSLHostnameVerifier(hostnameVerifier);
}
if (clientAuth) {
SSLConnectionSocketFactory sslCSF = new SSLConnectionSocketFactory(ssl,
new String[] { "TLSv1.1", "TLSv1.2" }, null, hostnameVerifier);
builder.setSSLSocketFactory(sslCSF);
}
return builder.build();
}
@@ -1,7 +1,8 @@
package timely.client.tcp;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.OutputStreamWriter;
import java.net.Socket;
import org.slf4j.Logger;
@@ -14,7 +15,7 @@ public class TcpClient implements AutoCloseable {
private final String host;
private final int port;
private Socket sock = null;
private PrintWriter out = null;
private BufferedWriter out = null;
private long connectTime = 0L;
private long backoff = 2000;
private int bufferSize = -1;
@@ -47,7 +48,6 @@ public class TcpClient implements AutoCloseable {
*
* @param metric
* newline terminated string representation of Timely metric
* @throws IOException
*/
public synchronized void write(String metric) throws IOException {
if (connect() != 0) {
@@ -60,7 +60,7 @@ public class TcpClient implements AutoCloseable {
}
}
public synchronized void flush() {
public synchronized void flush() throws IOException {
if (null != out) {
out.flush();
writesSinceFlush = 0;
@@ -94,19 +94,44 @@ public class TcpClient implements AutoCloseable {
}
private synchronized int connect() {
if (null == sock || !sock.isConnected() || out.checkError()) {
if (null == sock || !sock.isConnected()) {
if (System.currentTimeMillis() > (connectTime + backoff)) {
OutputStreamWriter osw = null;
try {
connectTime = System.currentTimeMillis();
sock = new Socket(host, port);
out = new PrintWriter(sock.getOutputStream(), false);
osw = new OutputStreamWriter(sock.getOutputStream());
out = new BufferedWriter(osw);
backoff = 2000;
LOG.trace("Connected to Timely at {}:{}", host, port);
} catch (IOException e) {
} catch (Exception e) {
LOG.error("Error connecting to Timely at {}:" + host + ":" + port + ". Error: " + e.getMessage());
if (sock != null) {
try {
sock.close();
} catch (IOException e1) {
LOG.error(e1.getMessage());
} finally {
sock = null;
}
}
if (osw != null) {
try {
osw.close();
} catch (IOException e1) {
LOG.error(e1.getMessage());
}
}
if (out != null) {
try {
out.close();
} catch (IOException e1) {
LOG.error(e1.getMessage());
} finally {
out = null;
}
}
backoff = backoff * 2;
sock = null;
out = null;
LOG.info("Will retry connection in {} ms.", backoff);
return -1;
}
@@ -1,5 +1,8 @@
package timely.client.websocket;
import java.util.List;
import java.util.Map;
import javax.websocket.CloseReason;
import javax.websocket.Endpoint;
import javax.websocket.EndpointConfig;
@@ -35,4 +38,6 @@ public abstract class ClientHandler extends Endpoint {
LOG.error("Error occurred on Websocket session" + session.getId(), error);
}
public void beforeRequest(Map<String, List<String>> headers) {
}
}
@@ -14,13 +14,15 @@ import org.apache.http.cookie.Cookie;
public class TimelyEndpointConfig implements ClientEndpointConfig {
private String sessionCookie = null;
private final ClientHandler clientEndpoint;
private static final String FORMAT = "%s=%s;";
public TimelyEndpointConfig(Cookie sessionCookie) {
public TimelyEndpointConfig(ClientHandler clientEndpoint, Cookie sessionCookie) {
if (null != sessionCookie) {
this.sessionCookie = String.format(FORMAT, sessionCookie.getName(), sessionCookie.getValue());
}
this.clientEndpoint = clientEndpoint;
}
@Override
@@ -58,6 +60,7 @@ public class TimelyEndpointConfig implements ClientEndpointConfig {
if (null != sessionCookie) {
headers.put("Cookie", Collections.singletonList(sessionCookie));
}
TimelyEndpointConfig.this.clientEndpoint.beforeRequest(headers);
}
};
}
@@ -40,6 +40,7 @@ public class WebSocketClient implements AutoCloseable {
private final int timelyHttpsPort;
private final int timelyWssPort;
private final boolean doLogin;
private final boolean clientAuth;
private final String timelyUsername;
private final String timelyPassword;
private final boolean hostVerificationEnabled;
@@ -51,13 +52,14 @@ public class WebSocketClient implements AutoCloseable {
protected volatile boolean closed = true;
protected WebSocketClient(SSLContext ssl, String timelyHostname, int timelyHttpsPort, int timelyWssPort,
boolean doLogin, String timelyUsername, String timelyPassword, boolean hostVerificationEnabled,
int bufferSize) {
boolean clientAuth, boolean doLogin, String timelyUsername, String timelyPassword,
boolean hostVerificationEnabled, int bufferSize) {
this.ssl = ssl;
this.timelyHostname = timelyHostname;
this.timelyHttpsPort = timelyHttpsPort;
this.timelyWssPort = timelyWssPort;
this.doLogin = doLogin;
this.clientAuth = clientAuth;
this.timelyUsername = timelyUsername;
this.timelyPassword = timelyPassword;
this.hostVerificationEnabled = hostVerificationEnabled;
@@ -67,28 +69,28 @@ public class WebSocketClient implements AutoCloseable {
Preconditions.checkNotNull(timelyHttpsPort, "%s must be supplied", "Timely HTTPS port");
Preconditions.checkNotNull(timelyWssPort, "%s must be supplied", "Timely WSS port");
if (doLogin && ((StringUtils.isEmpty(timelyUsername) && !StringUtils.isEmpty(timelyPassword)
|| (!StringUtils.isEmpty(timelyUsername) && StringUtils.isEmpty(timelyPassword))))) {
throw new IllegalArgumentException("Both Timely username and password must be empty or non-empty");
if (doLogin && !clientAuth) {
if ((StringUtils.isEmpty(timelyUsername) && !StringUtils.isEmpty(timelyPassword)
|| (!StringUtils.isEmpty(timelyUsername) && StringUtils.isEmpty(timelyPassword)))) {
throw new IllegalArgumentException("Both Timely username and password must be empty or non-empty");
}
}
}
protected WebSocketClient(String timelyHostname, int timelyHttpsPort, int timelyWssPort, boolean doLogin,
String timelyUsername, String timelyPassword, String keyStoreFile, String keyStoreType, String keyStorePass,
String trustStoreFile, String trustStoreType, String trustStorePass, boolean hostVerificationEnabled,
int bufferSize) {
protected WebSocketClient(String timelyHostname, int timelyHttpsPort, int timelyWssPort, boolean clientAuth,
boolean doLogin, String timelyUsername, String timelyPassword, String keyStoreFile, String keyStoreType,
String keyStorePass, String trustStoreFile, String trustStoreType, String trustStorePass,
boolean hostVerificationEnabled, int bufferSize) {
this(HttpClient.getSSLContext(trustStoreFile, trustStoreType, trustStorePass, keyStoreFile, keyStoreType,
keyStorePass), timelyHostname, timelyHttpsPort, timelyWssPort, doLogin, timelyUsername, timelyPassword,
hostVerificationEnabled, bufferSize);
keyStorePass), timelyHostname, timelyHttpsPort, timelyWssPort, clientAuth, doLogin, timelyUsername,
timelyPassword, hostVerificationEnabled, bufferSize);
}
public void open(ClientHandler clientEndpoint) throws IOException, DeploymentException, URISyntaxException {
Cookie sessionCookie = null;
if (doLogin) {
BasicCookieStore cookieJar = new BasicCookieStore();
try (CloseableHttpClient client = HttpClient.get(ssl, cookieJar, hostVerificationEnabled)) {
try (CloseableHttpClient client = HttpClient.get(ssl, cookieJar, hostVerificationEnabled, clientAuth)) {
String target = "https://" + timelyHostname + ":" + timelyHttpsPort + "/login";
@@ -140,8 +142,8 @@ public class WebSocketClient implements AutoCloseable {
webSocketClient.getProperties().put(ClientProperties.SSL_ENGINE_CONFIGURATOR, sslEngine);
webSocketClient.getProperties().put(ClientProperties.INCOMING_BUFFER_SIZE, bufferSize);
String wssPath = "wss://" + timelyHostname + ":" + timelyWssPort + "/websocket";
session = webSocketClient.connectToServer(clientEndpoint, new TimelyEndpointConfig(sessionCookie),
new URI(wssPath));
session = webSocketClient.connectToServer(clientEndpoint,
new TimelyEndpointConfig(clientEndpoint, sessionCookie), new URI(wssPath));
final ByteBuffer pingData = ByteBuffer.allocate(0);
webSocketClient.getScheduledExecutorService().scheduleAtFixedRate(() -> {
@@ -27,21 +27,21 @@ public class WebSocketSubscriptionClient extends WebSocketClient {
private final String subscriptionId;
public WebSocketSubscriptionClient(SSLContext ssl, String timelyHostname, int timelyHttpsPort, int timelyWssPort,
boolean doLogin, String timelyUsername, String timelyPassword, boolean hostVerificationEnabled,
int bufferSize) {
super(ssl, timelyHostname, timelyHttpsPort, timelyWssPort, doLogin, timelyUsername, timelyPassword,
boolean clientAuth, boolean doLogin, String timelyUsername, String timelyPassword,
boolean hostVerificationEnabled, int bufferSize) {
super(ssl, timelyHostname, timelyHttpsPort, timelyWssPort, clientAuth, doLogin, timelyUsername, timelyPassword,
hostVerificationEnabled, bufferSize);
subscriptionId = UUID.randomUUID().toString();
LOG.trace("Created WebSocketClient with subscriptionId {}", this.subscriptionId);
}
public WebSocketSubscriptionClient(String timelyHostname, int timelyHttpsPort, int timelyWssPort, boolean doLogin,
String timelyUsername, String timelyPassword, String keyStoreFile, String keyStoreType, String keyStorePass,
String trustStoreFile, String trustStoreType, String trustStorePass, boolean hostVerificationEnabled,
int bufferSize) {
super(timelyHostname, timelyHttpsPort, timelyWssPort, doLogin, timelyUsername, timelyPassword, keyStoreFile,
keyStoreType, keyStorePass, trustStoreFile, trustStoreType, trustStorePass, hostVerificationEnabled,
bufferSize);
public WebSocketSubscriptionClient(String timelyHostname, int timelyHttpsPort, int timelyWssPort,
boolean clientAuth, boolean doLogin, String timelyUsername, String timelyPassword, String keyStoreFile,
String keyStoreType, String keyStorePass, String trustStoreFile, String trustStoreType,
String trustStorePass, boolean hostVerificationEnabled, int bufferSize) {
super(timelyHostname, timelyHttpsPort, timelyWssPort, clientAuth, doLogin, timelyUsername, timelyPassword,
keyStoreFile, keyStoreType, keyStorePass, trustStoreFile, trustStoreType, trustStorePass,
hostVerificationEnabled, bufferSize);
subscriptionId = UUID.randomUUID().toString();
LOG.trace("Created WebSocketClient with subscriptionId {}", this.subscriptionId);
}
@@ -2,7 +2,6 @@ package timely.model;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.HashSet;
import java.util.Set;
@@ -62,7 +61,6 @@ public interface ObjectSizeOf {
* ObjectSizeOf interface and if implemented on the object will use that.
* Otherwise it will do a simple navigation of the fields using reflection.
*
* @param o
* @return an approximation of the object size
*/
public static long getObjectSize(Object o, boolean useSizeInBytesMethod, boolean debug) {
@@ -88,15 +86,8 @@ public interface ObjectSizeOf {
visited.add(oi);
try {
if (useSizeInBytesMethod) {
try {
if (o instanceof ObjectSizeOf) {
size = ((ObjectSizeOf) o).sizeInBytes();
} else {
Method sizeInBytes = o.getClass().getMethod("sizeInBytes", (Class<?>[]) null);
size = (Long) sizeInBytes.invoke(o);
}
} catch (Throwable t) {
// ok, lets do this the hard way...
if (o instanceof ObjectSizeOf) {
size = ((ObjectSizeOf) o).sizeInBytes();
}
}
if (size == 0) {
@@ -77,8 +77,6 @@ public class Tag implements Comparable<Tag>, Serializable {
/**
* Joins key and value into key=value
*
* @return
*/
public String join() {
return tagParser.combine(this);
@@ -47,7 +47,6 @@ public class TagListParser implements Parser<List<Tag>>, Combiner<Collection<Tag
* An alternative combiner for a map of string to string instead of a list of
* tags
*
* @param tags
* @return string representation
*/
public String combine(Map<String, String> tags) {
@@ -23,7 +23,7 @@ def pivot(df, metric, groupByColumn=None):
return dataFrame
def resample(df, sample, how='mean'):
def resample(df, sample, how='mean', interpolate=True, fill=None):
dataFrame = pandas.DataFrame(df, copy=True)
if dataFrame is not None:
@@ -16,14 +16,15 @@ utc = UTC()
class TimelyWebSocketClient(WebSocketClient):
def __init__(self, hostport, metric, tags, startTime, endTime, connect_timeout=WebSocketClient.DEFAULT_CONNECT_TIMEOUT,
request_timeout=WebSocketClient.DEFAULT_REQUEST_TIMEOUT):
request_timeout=WebSocketClient.DEFAULT_REQUEST_TIMEOUT, client_cert=None, client_key=None, ca_certs=None, validate_cert=False):
self.metric = metric
self.tags = tags
self.startTime = startTime
self.endTime = endTime
self.subscriptionId = random.randint(1000000000, 9999999999)
WebSocketClient.__init__(self, hostport, '/websocket', {'metric' : metric}, connect_timeout, request_timeout)
WebSocketClient.__init__(self, hostport, '/websocket', {'metric' : metric}, connect_timeout, request_timeout,
client_cert=client_cert, client_key=client_key, ca_certs=ca_certs, validate_cert=validate_cert)
def _on_message(self, msg):
# implement in subclass
@@ -72,7 +73,8 @@ class TimelyMetric(TimelyWebSocketClient):
client = None
endtime = None
def __init__(self, hostport, metric, tags, beginStr, endStr, period, sample):
def __init__(self, hostport, metric, tags, beginStr, endStr, period, sample,
client_cert=None, client_key=None, ca_certs=None, validate_cert=False):
self.series = pandas.Series()
self.metric = metric
self.tags = tags
@@ -82,7 +84,8 @@ class TimelyMetric(TimelyWebSocketClient):
self.data = []
self.timeDateRange = TimeDateRange(beginStr, endStr, period)
TimelyWebSocketClient.__init__(self, hostport, metric, tags, self.timeDateRange.getBeginMs(), self.timeDateRange.getEndMs())
TimelyWebSocketClient.__init__(self, hostport, metric, tags, self.timeDateRange.getBeginMs(), self.timeDateRange.getEndMs(),
client_cert=client_cert, client_key=client_key, ca_certs=ca_certs, validate_cert=validate_cert)
def debugOn(self):
self.debug = True
@@ -15,10 +15,14 @@ class WebSocketClient():
APPLICATION_JSON = 'application/json'
def __init__(self, hostport, path, params, connect_timeout=DEFAULT_CONNECT_TIMEOUT,
request_timeout=DEFAULT_REQUEST_TIMEOUT):
request_timeout=DEFAULT_REQUEST_TIMEOUT, client_cert=None, client_key=None, ca_certs=None, validate_cert=False):
self.connect_timeout = connect_timeout
self.request_timeout = request_timeout
self.client_cert=client_cert
self.client_key=client_key
self.ca_certs=ca_certs
self.validate_cert=validate_cert
self.url = 'wss://' + hostport
if (path is not None and len(path) > 0):
if (path.startswith('/')):
@@ -39,7 +43,8 @@ class WebSocketClient():
"""Connect to the server.
:param str url: server URL.
"""
request = websocket.httpclient.HTTPRequest(self.url, connect_timeout=self.connect_timeout, request_timeout=self.request_timeout, validate_cert=False)
request = websocket.httpclient.HTTPRequest(self.url, connect_timeout=self.connect_timeout, request_timeout=self.request_timeout, validate_cert=self.validate_cert,
client_cert=self.client_cert, client_key=self.client_key, ca_certs=self.ca_certs)
websocket.websocket_connect(request, io_loop=ioloop, callback=self._connect_callback)
def send(self, data):
@@ -5,7 +5,9 @@ import os
from timely.TimelyAnalyticConfiguration import TimelyAnalyticConfiguration
hostport = "127.0.0.1:5244"
timelyMetric = timely.TimelyMetric.TimelyMetric(hostport, 'timely.keys.metric.inserted', None, None, None, '24 hours', None).fetch()
client_cert = None
client_key = None
timelyMetric = timely.TimelyMetric.TimelyMetric(hostport, 'timely.keys.metric.inserted', None, None, None, '24 hours', None, client_cert=client_cert, client_key=client_key).fetch()
print(timelyMetric.getDataFrame())
exit()
+5
View File
@@ -5,4 +5,9 @@
<Method name="connect" />
<Class name="timely.client.tcp.TcpClient"/>
</Match>
<Match>
<!-- Must ignore these everywhere, because of a javac byte code generation bug -->
<!-- https://github.com/spotbugs/spotbugs/issues/756 -->
<Bug pattern="RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE" />
</Match>
</FindBugsFilter>
@@ -47,9 +47,9 @@ public class SerializationTest {
testSerialization(remove);
}
@SuppressWarnings("unchecked")
private <T> void testSerialization(T o) throws Exception {
String ser = mapper.writeValueAsString(o);
@SuppressWarnings("unchecked")
T des = mapper.readValue(ser, (Class<T>) o.getClass());
Assert.assertEquals(o, des);
}
+51 -51
View File
@@ -1,51 +1,51 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>collectd-nsq-plugin</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>collectd-plugin-parent</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<artifactSet>
<includes />
</artifactSet>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<id>shade</id>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>collectd-nsq-plugin</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>collectd-plugin-parent</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<artifactSet>
<includes />
</artifactSet>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<id>shade</id>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
@@ -10,7 +10,7 @@ import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.collectd.api.OConfigItem;
public class PooledCloseableHttpClientFactory implements PooledObjectFactory {
public class PooledCloseableHttpClientFactory implements PooledObjectFactory<CloseableHttpClient> {
private int connectionRequestTimeout = 10000; // 10 seconds in milliseconds
private int connectTimeout = 30000; // 30 seconds in milliseconds
@@ -43,19 +43,19 @@ public class PooledCloseableHttpClientFactory implements PooledObjectFactory {
}
@Override
public PooledObject makeObject() throws Exception {
public PooledObject<CloseableHttpClient> makeObject() throws Exception {
RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(connectionRequestTimeout)
.setConnectTimeout(connectTimeout).setSocketTimeout(socketTimeout).build();
HttpClientBuilder builder = HttpClientBuilder.create();
builder.setDefaultRequestConfig(requestConfig);
builder.setConnectionTimeToLive(connectionTimeToLive, TimeUnit.MILLISECONDS);
return new DefaultPooledObject(builder.build());
return new DefaultPooledObject<>(builder.build());
}
@Override
public void destroyObject(PooledObject pooledObject) throws Exception {
public void destroyObject(PooledObject<CloseableHttpClient> pooledObject) throws Exception {
try {
((CloseableHttpClient) pooledObject.getObject()).close();
pooledObject.getObject().close();
} catch (Exception e) {
// do nothing
} finally {
@@ -65,17 +65,17 @@ public class PooledCloseableHttpClientFactory implements PooledObjectFactory {
}
@Override
public boolean validateObject(PooledObject pooledObject) {
public boolean validateObject(PooledObject<CloseableHttpClient> pooledObject) {
return pooledObject.getObject() != null;
}
@Override
public void activateObject(PooledObject pooledObject) throws Exception {
public void activateObject(PooledObject<CloseableHttpClient> pooledObject) throws Exception {
// do nothing
}
@Override
public void passivateObject(PooledObject pooledObject) throws Exception {
public void passivateObject(PooledObject<CloseableHttpClient> pooledObject) throws Exception {
// do nothing
}
@@ -47,6 +47,7 @@ public class WriteNSQPlugin extends CollectDPluginParent
Collectd.registerWrite(WriteNSQPlugin.class.getName(), this);
}
@Override
public int write(ValueList vl) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream out = new DataOutputStream(baos);
@@ -110,15 +111,17 @@ public class WriteNSQPlugin extends CollectDPluginParent
}
}
@Override
public int shutdown() {
Collectd.logInfo("Shutting down connection to NSQ at " + host + ":" + port);
clientPool.close();
return 0;
}
@Override
public int config(OConfigItem config) {
super.config(config);
GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
GenericObjectPoolConfig<CloseableHttpClient> poolConfig = new GenericObjectPoolConfig<>();
// use max size for maxTotal and maxIdle
// no need to activate or passivate
poolConfig.setMaxTotal(POOL_MAX_SIZE);
@@ -156,7 +159,7 @@ public class WriteNSQPlugin extends CollectDPluginParent
Collectd.logError("NSQ host, port, and topic must be configured");
return -1;
}
this.clientPool = new GenericObjectPool(pooledCloseableHttpClientFactory, poolConfig);
this.clientPool = new GenericObjectPool<>(pooledCloseableHttpClientFactory, poolConfig);
this.endpoint = "http://" + host + ":" + port + "/mpub?topic=" + topic;
return 0;
}
+32 -32
View File
@@ -1,32 +1,32 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>collectd-plugin-parent</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-easymock-release-full</artifactId>
<classifier>full</classifier>
<scope>test</scope>
</dependency>
</dependencies>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>collectd-plugin-parent</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-easymock-release-full</artifactId>
<classifier>full</classifier>
<scope>test</scope>
</dependency>
</dependencies>
</project>
+68 -68
View File
@@ -1,68 +1,68 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>collectd-timely-plugin</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>collectd-plugin-parent</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-easymock-release-full</artifactId>
<classifier>full</classifier>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<artifactSet>
<includes />
</artifactSet>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<id>shade</id>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>collectd-timely-plugin</artifactId>
<dependencies>
<dependency>
<groupId>collectd</groupId>
<artifactId>collectd-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>collectd-plugin-parent</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.jcip</groupId>
<artifactId>jcip-annotations</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.powermock</groupId>
<artifactId>powermock-easymock-release-full</artifactId>
<classifier>full</classifier>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<configuration>
<artifactSet>
<includes />
</artifactSet>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<id>shade</id>
<goals>
<goal>shade</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
@@ -8,7 +8,7 @@ import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.collectd.api.Collectd;
import org.collectd.api.OConfigItem;
public class PooledSocketFactory implements PooledObjectFactory {
public class PooledSocketFactory implements PooledObjectFactory<Socket> {
private String host = null;
private int port = 0;
@@ -81,12 +81,12 @@ public class PooledSocketFactory implements PooledObjectFactory {
}
@Override
public PooledObject makeObject() throws Exception {
return new DefaultPooledObject(connect());
public PooledObject<Socket> makeObject() throws Exception {
return new DefaultPooledObject<>(connect());
}
@Override
public void destroyObject(PooledObject pooledObject) throws Exception {
public void destroyObject(PooledObject<Socket> pooledObject) throws Exception {
try {
Collectd.logDebug("Shutting down connection to Timely at " + host + ":" + port);
Socket socket = (Socket) pooledObject.getObject();
@@ -103,7 +103,7 @@ public class PooledSocketFactory implements PooledObjectFactory {
}
@Override
public boolean validateObject(PooledObject pooledObject) {
public boolean validateObject(PooledObject<Socket> pooledObject) {
if (connectionTimeToLive > -1) {
if (System.currentTimeMillis() > pooledObject.getCreateTime() + connectionTimeToLive) {
return false;
@@ -114,12 +114,12 @@ public class PooledSocketFactory implements PooledObjectFactory {
}
@Override
public void activateObject(PooledObject pooledObject) throws Exception {
public void activateObject(PooledObject<Socket> pooledObject) throws Exception {
// do nothing
}
@Override
public void passivateObject(PooledObject pooledObject) throws Exception {
public void passivateObject(PooledObject<Socket> pooledObject) throws Exception {
// do nothing
}
}
@@ -86,13 +86,13 @@ public class WriteTimelyPlugin extends CollectDPluginParent
PooledSocketFactory socketFactory = new PooledSocketFactory();
retval = socketFactory.config(config);
if (retval == 0) {
GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
GenericObjectPoolConfig<Socket> poolConfig = new GenericObjectPoolConfig<>();
// use max size for maxTotal and maxIdle
// no need to activate and passivate
poolConfig.setMaxTotal(POOL_MAX_SIZE);
poolConfig.setMaxIdle(POOL_MAX_SIZE);
poolConfig.setTestOnReturn(true);
socketPool = new GenericObjectPool(socketFactory, poolConfig);
socketPool = new GenericObjectPool<>(socketFactory, poolConfig);
}
return retval;
}
+17 -17
View File
@@ -1,17 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>timely-common</artifactId>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>timely-common</artifactId>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
+42
View File
@@ -0,0 +1,42 @@
#!/bin/bash
if [[ `uname` == "Darwin" ]]; then
THIS_SCRIPT=`python -c 'import os,sys; print os.path.realpath(sys.argv[1])' $0`
TCNATIVE_SUFFIX="jnilib"
else
THIS_SCRIPT=`readlink -f $0`
TCNATIVE_SUFFIX="so"
fi
THIS_DIR="${THIS_SCRIPT%/*}"
NATIVE_DIR="${THIS_DIR}/META-INF/native"
BASE_DIR=${THIS_DIR}/..
TMP_DIR="${BASE_DIR}/tmp"
CONF_DIR="${BASE_DIR}/conf"
LIB_DIR="${BASE_DIR}/lib"
NUM_SERVER_THREADS=4
if [[ -e ${TMP_DIR} ]]; then
rm -rf ${TMP_DIR}
fi
mkdir ${TMP_DIR}
if [[ -e ${NATIVE_DIR} ]]; then
rm -rf ${NATIVE_DIR}
fi
mkdir -p ${NATIVE_DIR}
pushd ${BASE_DIR}/bin
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-tcnative-boringssl-static*.jar META-INF/native/libnetty_tcnative_linux_x86_64.${TCNATIVE_SUFFIX}
$JAVA_HOME/bin/jar xf ${LIB_DIR}/netty-all*.jar META-INF/native/libnetty_transport_native_epoll_x86_64.${TCNATIVE_SUFFIX}
popd
export CLASSPATH="${CONF_DIR}:${LIB_DIR}/*:${HADOOP_CONF_DIR}"
JVM_ARGS="-Xmx1G -Xms1G -Dio.netty.eventLoopThreads=${NUM_SERVER_THREADS} Dlog4j.configurationFile=${CONF_DIR}/log4j2-spring.xml"
JVM_ARGS="${JVM_ARGS} -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector"
JVM_ARGS="${JVM_ARGS} -Djava.library.path=${NATIVE_DIR}"
#JVM_ARGS="${JVM_ARGS} -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
echo "$JAVA_HOME/bin/java ${JVM_ARGS} timely.grafana.auth.GrafanaAuth --spring.config.name=grafana-auth"
$JAVA_HOME/bin/java ${JVM_ARGS} timely.grafana.auth.GrafanaAuth --spring.config.name=grafana-auth
+55
View File
@@ -0,0 +1,55 @@
grafana-auth:
grafana:
host: localhost
port: 3001
security:
session-max-age: 86400
cache-expiration-minutes: 60
cache-refresh-minutes: 5
allow-anonymous-http-access: true
allow-anonymous-ws-access: true
serverSsl:
certificate-file:
key-file:
#key-password:
use-generated-keypair: true
trust-store-file:
use-openssl: true
#use-ciphers:
clientSsl:
keyFile:
keyType: pkcs12
#keyPassword:
trustStoreFile:
trustStoreType: jks
#trustStorePassword:
http:
host: localhost
ip: 127.0.0.1
port: 3000
redirect-path: /secure-me
strict-transport-max-age: 608400
cors:
allow-any-origin: false
allow-null-origin: false
allow-credentials: true
allowed-origins:
allowed-methods: DELETE,GET,HEAD,OPTIONS,PUT,POST
allowed-headers: content-type
httpClientPool:
minIdlePerKey: 0
maxIdlePerKey: 40
maxTotalPerKey: 40
maxTotal: -1
lifo: true
fairness: false
maxWaitMillis: -1
minEvictableIdleTimeMillis: 1800000
evictorShutdownTimeoutMillis: 10000
softMinEvictableIdleTimeMillis: -1
numTestsPerEvictionRun: 3
timeBetweenEvictionRunsMillis: -1
blockWhenExhausted: true
jmxEnabled: true
testOnBorrow: false
testOnReturn: false
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" monitorInterval="60">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="error" includeLocation="false">
<AppenderRef ref="Console" level="error" />
</Root>
</Loggers>
</Configuration>
+29
View File
@@ -0,0 +1,29 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" monitorInterval="60">
<Properties>
<Property name="PID">????</Property>
<Property name="LOG_PATTERN">%clr{%d{yyyy-MM-dd HH:mm:ss.SSS}}{faint} %clr{%5p} %clr{${sys:PID}}{magenta} %clr{---}{faint} %clr{[%15.15t]}{faint} %clr{%-40.40c{1.}}{cyan} %clr{:}{faint} %m%n%wEx</Property>
</Properties>
<Appenders>
<RollingRandomAccessFile name="Rolling" fileName="/srv/logs/timely/grafana-auth.log"
filePattern="/srv/logs/timely/$${date:yyyy-MM}/grafana-auth-%d{MM-dd-yyyy}-%i.log.gz">
<PatternLayout pattern="${LOG_PATTERN}"/>
<Policies>
<SizeBasedTriggeringPolicy size="1024 MB" />
</Policies>
<DefaultRolloverStrategy max="10" />
</RollingRandomAccessFile>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${LOG_PATTERN}"/>
</Console>
</Appenders>
<Loggers>
<Root level="trace" includeLocation="false">
<AppenderRef ref="Console" level="warn" />
<AppenderRef ref="Rolling" level="trace" />
</Root>
<Logger name="timely" additivity="false">
<AppenderRef ref="Console" level="debug" />
</Logger>
</Loggers>
</Configuration>
+71
View File
@@ -0,0 +1,71 @@
<beans:beans xmlns="http://www.springframework.org/schema/security"
xmlns:beans="http://www.springframework.org/schema/beans"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/security
http://www.springframework.org/schema/security/spring-security.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<beans:bean id="x509PrincipalExtractor" class="org.springframework.security.web.authentication.preauth.x509.SubjectDnX509PrincipalExtractor">
<beans:property name="subjectDnRegex" value="CN=(.*?)(?:,|$)"/>
</beans:bean>
<beans:bean id="timelyUserDetailsService" class="timely.auth.UserDetailsService">
<beans:property name="users">
<util:list>
<beans:bean class="timely.auth.TimelyUser">
<beans:constructor-arg name="subjectDn">
<beans:value>CN=example.com</beans:value>
</beans:constructor-arg>
<beans:constructor-arg name="auths">
<util:list>
<beans:value>D</beans:value>
<beans:value>E</beans:value>
<beans:value>F</beans:value>
</util:list>
</beans:constructor-arg>
<beans:constructor-arg name="roles">
<util:list>
<beans:value>G</beans:value>
<beans:value>H</beans:value>
<beans:value>I</beans:value>
</util:list>
</beans:constructor-arg>
</beans:bean>
</util:list>
</beans:property>
</beans:bean>
<beans:bean id="x509AuthorizationProvider" class="org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationProvider">
<beans:property name="preAuthenticatedUserDetailsService" ref="timelyUserDetailsService" />
</beans:bean>
<user-service id="usernamePasswordService">
<user name="test" password="test1" authorities="A,B,C" />
</user-service>
<beans:bean id="noOpPasswordEncoder" class="org.springframework.security.crypto.password.NoOpPasswordEncoder"/>
<beans:bean id="UserNamePasswordAuthenticationProvider" class="org.springframework.security.authentication.dao.DaoAuthenticationProvider">
<beans:property name="userDetailsService" ref="usernamePasswordService"/>
<beans:property name="passwordEncoder" ref="noOpPasswordEncoder"/>
</beans:bean>
<authentication-manager alias="authenticationManager">
<!-- X509 PreAuthenticated authorization provider -->
<authentication-provider ref="x509AuthorizationProvider"/>
<!-- Basic authorization provider, validates provided username and password -->
<authentication-provider ref="UserNamePasswordAuthenticationProvider" />
</authentication-manager>
<util:list id="requiredRoles" value-type="java.lang.String">
<beans:value></beans:value>
</util:list>
<util:list id="requiredAuths" value-type="java.lang.String">
<beans:value></beans:value>
</util:list>
</beans:beans>
+295
View File
@@ -0,0 +1,295 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>grafana-auth</artifactId>
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jdk8</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>
<dependency>
<groupId>com.google.flatbuffers</groupId>
<artifactId>flatbuffers-java</artifactId>
</dependency>
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
</dependency>
<dependency>
<groupId>io.github.lukehutch</groupId>
<artifactId>fast-classpath-scanner</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</dependency>
<dependency>
<groupId>net.sourceforge.javacsv</groupId>
<artifactId>javacsv</artifactId>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-core</artifactId>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.accumulo</groupId>
<artifactId>accumulo-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-framework</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-recipes</artifactId>
</dependency>
<dependency>
<groupId>org.apache.curator</groupId>
<artifactId>curator-x-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<exclusions>
<exclusion>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-websocket</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-balancer</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-client</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-common</artifactId>
</dependency>
<dependency>
<groupId>timely</groupId>
<artifactId>timely-server</artifactId>
</dependency>
<dependency>
<groupId>com.lmax</groupId>
<artifactId>disruptor</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.javassist</groupId>
<artifactId>javassist</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-test-autoconfigure</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>assemble</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>src/main/dist/assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<goals>
<goal>copy-dependencies</goal>
</goals>
<phase>prepare-package</phase>
<configuration>
<excludeArtifactIds>junit</excludeArtifactIds>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
<execution>
<id>unpack-native</id>
<goals>
<goal>unpack</goal>
</goals>
<phase>process-test-classes</phase>
<configuration>
<artifactItems>
<artifactItem>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>${version.netty-tcnative}</version>
<type>jar</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}/native</outputDirectory>
<includes>**/*.so</includes>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<execution>
<id>add-source</id>
<goals>
<goal>add-source</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<sources>
<source>src/main/generated-java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<extensions>
<!-- Use os-maven-plugin to initialize the "os.detected" properties -->
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
</extension>
</extensions>
</build>
</project>
+55
View File
@@ -0,0 +1,55 @@
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">
<id>dist</id>
<formats>
<format>tar.gz</format>
</formats>
<fileSets>
<fileSet>
<directory>${project.basedir}</directory>
<outputDirectory></outputDirectory>
<includes>
<include>README*</include>
<include>LICENSE*</include>
<include>NOTICE*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.build.directory}</directory>
<outputDirectory>lib</outputDirectory>
<includes>
<include>timely*.jar</include>
<include>grafana*.jar</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.build.directory}/lib</directory>
<outputDirectory>lib</outputDirectory>
<includes>
<include>*.jar</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.basedir}/bin</directory>
<outputDirectory>bin</outputDirectory>
<includes>
<include>**/*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.basedir}/conf</directory>
<outputDirectory>conf</outputDirectory>
<includes>
<include>*</include>
</includes>
</fileSet>
<fileSet>
<directory>${project.basedir}/src/main/resources/webapp</directory>
<outputDirectory>bin/webapp</outputDirectory>
<includes>
<include>*</include>
</includes>
</fileSet>
</fileSets>
</assembly>
@@ -0,0 +1,322 @@
package timely.grafana.auth;
import java.io.File;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import javax.net.ssl.SSLContext;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.PooledByteBufAllocator;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.epoll.EpollEventLoopGroup;
import io.netty.channel.epoll.EpollServerSocketChannel;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.ServerSocketChannel;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.HttpContentCompressor;
import io.netty.handler.codec.http.HttpContentDecompressor;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpRequestDecoder;
import io.netty.handler.codec.http.HttpResponseEncoder;
import io.netty.handler.logging.LoggingHandler;
import io.netty.handler.ssl.ClientAuth;
import io.netty.handler.ssl.OpenSslServerContext;
import io.netty.handler.ssl.OpenSslServerSessionContext;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.util.SelfSignedCertificate;
import io.netty.handler.stream.ChunkedWriteHandler;
import io.netty.util.internal.SystemPropertyUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.WebApplicationType;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.context.ConfigurableApplicationContext;
import timely.auth.AuthCache;
import timely.balancer.configuration.ClientSsl;
import timely.balancer.connection.http.HttpClientPool;
import timely.client.http.HttpClient;
import timely.configuration.ServerSsl;
import timely.grafana.auth.configuration.GrafanaAuthConfiguration;
import timely.grafana.auth.configuration.SpringBootstrap;
import timely.grafana.auth.netty.http.GrafanaRelayHandler;
import timely.grafana.auth.netty.http.GrafanaRequestDecoder;
import timely.netty.http.HttpStaticFileServerHandler;
import timely.netty.http.NonSslRedirectHandler;
import timely.netty.http.TimelyExceptionHandler;
import timely.netty.http.auth.X509LoginRequestHandler;
public class GrafanaAuth {
private static final Logger LOG = LoggerFactory.getLogger(GrafanaAuth.class);
private static final int EPOLL_MIN_MAJOR_VERSION = 2;
private static final int EPOLL_MIN_MINOR_VERSION = 6;
private static final int EPOLL_MIN_PATCH_VERSION = 32;
private static final String OS_NAME = "os.name";
private static final String OS_VERSION = "os.version";
protected static final CountDownLatch LATCH = new CountDownLatch(1);
static ConfigurableApplicationContext applicationContext;
private final GrafanaAuthConfiguration grafanaAuthConfig;
private EventLoopGroup httpWorkerGroup = null;
private EventLoopGroup httpBossGroup = null;
protected Channel httpChannelHandle = null;
protected volatile boolean shutdown = false;
public GrafanaAuth(GrafanaAuthConfiguration grafanaAuthConfig) throws Exception {
this.grafanaAuthConfig = grafanaAuthConfig;
}
protected static ConfigurableApplicationContext initializeContext(String[] args) {
SpringApplicationBuilder builder = new SpringApplicationBuilder(SpringBootstrap.class);
builder.web(WebApplicationType.NONE);
builder.registerShutdownHook(false);
return builder.run(args);
}
private void shutdownHook() {
final Runnable shutdownRunner = () -> {
if (!shutdown) {
shutdown();
}
};
final Thread hook = new Thread(shutdownRunner, "shutdown-hook-thread");
Runtime.getRuntime().addShutdownHook(hook);
}
public void shutdown() {
List<ChannelFuture> channelFutures = new ArrayList<>();
if (httpChannelHandle != null) {
LOG.info("Closing httpChannelHandle");
channelFutures.add(httpChannelHandle.close());
}
// wait for the channels to shutdown
channelFutures.forEach(f -> {
try {
f.get();
} catch (final Exception e) {
LOG.error("Channel:" + f.channel().config() + " -> " + e.getMessage(), e);
}
});
int quietPeriod = grafanaAuthConfig.getShutdownQuietPeriod();
List<Future<?>> groupFutures = new ArrayList<>();
if (httpBossGroup != null) {
LOG.info("Shutting down httpBossGroup");
groupFutures.add(httpBossGroup.shutdownGracefully(quietPeriod, 10, TimeUnit.SECONDS));
}
if (httpWorkerGroup != null) {
LOG.info("Shutting down httpWorkerGroup");
groupFutures.add(httpWorkerGroup.shutdownGracefully(quietPeriod, 10, TimeUnit.SECONDS));
}
groupFutures.parallelStream().forEach(f -> {
try {
f.get();
} catch (final Exception e) {
LOG.error("Group:" + f.toString() + " -> " + e.getMessage(), e);
}
});
if (applicationContext != null) {
LOG.info("Closing applicationContext");
applicationContext.close();
}
this.shutdown = true;
LOG.info("Server shut down.");
}
private static boolean useEpoll() {
// Should we just return true if this is Linux and if we get an error
// during Epoll
// setup handle it there?
final String os = SystemPropertyUtil.get(OS_NAME).toLowerCase().trim();
final String[] version = SystemPropertyUtil.get(OS_VERSION).toLowerCase().trim().split("\\.");
if (os.startsWith("linux") && version.length >= 3) {
final int major = Integer.parseInt(version[0]);
if (major > EPOLL_MIN_MAJOR_VERSION) {
return true;
} else if (major == EPOLL_MIN_MAJOR_VERSION) {
final int minor = Integer.parseInt(version[1]);
if (minor > EPOLL_MIN_MINOR_VERSION) {
return true;
} else if (minor == EPOLL_MIN_MINOR_VERSION) {
final int patch = Integer.parseInt(version[2].substring(0, 2));
return patch >= EPOLL_MIN_PATCH_VERSION;
} else {
return false;
}
} else {
return false;
}
} else {
return false;
}
}
protected SslContext createSSLContext(GrafanaAuthConfiguration config) throws Exception {
ServerSsl sslCfg = config.getSecurity().getServerSsl();
Boolean generate = sslCfg.isUseGeneratedKeypair();
SslContextBuilder ssl;
if (generate) {
LOG.warn("Using generated self signed server certificate");
Date begin = new Date();
Date end = new Date(begin.getTime() + 86400000);
SelfSignedCertificate ssc = new SelfSignedCertificate("localhost", begin, end);
ssl = SslContextBuilder.forServer(ssc.certificate(), ssc.privateKey());
} else {
String cert = sslCfg.getCertificateFile();
String key = sslCfg.getKeyFile();
String keyPass = sslCfg.getKeyPassword();
if (null == cert || null == key) {
throw new IllegalArgumentException("Check your SSL properties, something is wrong.");
}
ssl = SslContextBuilder.forServer(new File(cert), new File(key), keyPass);
}
ssl.ciphers(sslCfg.getUseCiphers());
// Can't set to REQUIRE because the CORS pre-flight requests will fail.
ssl.clientAuth(ClientAuth.OPTIONAL);
Boolean useOpenSSL = sslCfg.isUseOpenssl();
if (useOpenSSL) {
ssl.sslProvider(SslProvider.OPENSSL);
} else {
ssl.sslProvider(SslProvider.JDK);
}
String trustStore = sslCfg.getTrustStoreFile();
if (null != trustStore) {
if (!trustStore.isEmpty()) {
ssl.trustManager(new File(trustStore));
}
}
return ssl.build();
}
protected SSLContext createSSLClientContext(GrafanaAuthConfiguration config) throws Exception {
ClientSsl ssl = config.getSecurity().getClientSsl();
return HttpClient.getSSLContext(ssl.getTrustStoreFile(), ssl.getTrustStoreType(), ssl.getTrustStorePassword(),
ssl.getKeyFile(), ssl.getKeyType(), ssl.getKeyPassword());
}
public void run() throws Exception {
AuthCache.configure(grafanaAuthConfig.getSecurity());
final boolean useEpoll = useEpoll();
Class<? extends ServerSocketChannel> channelClass;
if (useEpoll) {
httpWorkerGroup = new EpollEventLoopGroup();
httpBossGroup = new EpollEventLoopGroup();
channelClass = EpollServerSocketChannel.class;
} else {
httpWorkerGroup = new NioEventLoopGroup();
httpBossGroup = new NioEventLoopGroup();
channelClass = NioServerSocketChannel.class;
}
LOG.info("Using channel class {}", channelClass.getSimpleName());
final int httpPort = grafanaAuthConfig.getHttp().getPort();
final String httpIp = grafanaAuthConfig.getHttp().getIp();
SslContext sslCtx = createSSLContext(grafanaAuthConfig);
if (sslCtx instanceof OpenSslServerContext) {
OpenSslServerContext openssl = (OpenSslServerContext) sslCtx;
String application = "GrafanaAuth_" + httpPort;
OpenSslServerSessionContext opensslCtx = openssl.sessionContext();
opensslCtx.setSessionCacheEnabled(true);
opensslCtx.setSessionCacheSize(128);
opensslCtx.setSessionIdContext(application.getBytes(StandardCharsets.UTF_8));
opensslCtx.setSessionTimeout(grafanaAuthConfig.getSecurity().getSessionMaxAge());
}
SSLContext clientSSLContext = createSSLClientContext(grafanaAuthConfig);
final ServerBootstrap httpServer = new ServerBootstrap();
httpServer.group(httpBossGroup, httpWorkerGroup);
httpServer.channel(channelClass);
httpServer.handler(new LoggingHandler());
HttpClientPool httpClientPool = new HttpClientPool(grafanaAuthConfig.getSecurity(), grafanaAuthConfig.getHttp(),
clientSSLContext);
httpServer.childHandler(setupHttpChannel(grafanaAuthConfig, sslCtx, httpClientPool));
httpServer.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
httpServer.option(ChannelOption.SO_BACKLOG, 128);
httpServer.option(ChannelOption.SO_KEEPALIVE, true);
httpChannelHandle = httpServer.bind(httpIp, httpPort).sync().channel();
final String httpAddress = ((InetSocketAddress) httpChannelHandle.localAddress()).getAddress().getHostAddress();
shutdownHook();
LOG.info("Server started. Listening on {}:{} for HTTP traffic", httpAddress, httpPort);
}
public static void main(String[] args) throws Exception {
GrafanaAuth.applicationContext = GrafanaAuth.initializeContext(args);
GrafanaAuthConfiguration grafanaAuthConf = GrafanaAuth.applicationContext
.getBean(GrafanaAuthConfiguration.class);
GrafanaAuth grafanaAuth = new GrafanaAuth(grafanaAuthConf);
try {
grafanaAuth.run();
LATCH.await();
} catch (final InterruptedException e) {
LOG.info("Server shutting down.");
} catch (Exception e) {
LOG.error("Error running server.", e);
} finally {
try {
grafanaAuth.shutdown();
} catch (Exception e) {
System.exit(1);
}
}
}
protected ChannelHandler setupHttpChannel(GrafanaAuthConfiguration config, SslContext sslCtx,
HttpClientPool httpClientPool) {
return new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast("ssl", new NonSslRedirectHandler(config.getHttp(), sslCtx));
ch.pipeline().addLast("encoder", new HttpResponseEncoder());
ch.pipeline().addLast("decoder", new HttpRequestDecoder());
ch.pipeline().addLast("compressor", new HttpContentCompressor());
ch.pipeline().addLast("decompressor", new HttpContentDecompressor());
// high maximum contentLength so that grafana snapshots can be delivered
// might not be necessary if inbound chunking (while proxying) is handled
ch.pipeline().addLast("aggregator", new HttpObjectAggregator(2097152));
ch.pipeline().addLast("chunker", new ChunkedWriteHandler());
ch.pipeline().addLast("grafanaDecoder",
new GrafanaRequestDecoder(config.getSecurity(), config.getHttp()));
ch.pipeline().addLast("fileServer", new HttpStaticFileServerHandler());
ch.pipeline().addLast("login", new X509LoginRequestHandler(config.getSecurity(), config.getHttp()));
ch.pipeline().addLast("httpRelay", new GrafanaRelayHandler(config, httpClientPool));
ch.pipeline().addLast("error", new TimelyExceptionHandler()
.setIgnoreSslHandshakeErrors(config.getSecurity().getServerSsl().isUseGeneratedKeypair()));
}
};
}
}
@@ -0,0 +1,32 @@
package timely.grafana.auth.configuration;
public class Grafana {
private String host = "localhost";
private Integer port = 3000;
private String protocol = "http";
public void setHost(String host) {
this.host = host;
}
public String getHost() {
return host;
}
public void setPort(Integer port) {
this.port = port;
}
public Integer getPort() {
return port;
}
public void setProtocol(String protocol) {
this.protocol = protocol;
}
public String getProtocol() {
return protocol;
}
}
@@ -0,0 +1,54 @@
package timely.grafana.auth.configuration;
import javax.validation.Valid;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.NestedConfigurationProperty;
import org.springframework.stereotype.Component;
import org.springframework.validation.annotation.Validated;
import timely.balancer.configuration.BalancerHttp;
import timely.balancer.configuration.BalancerSecurity;
@Validated
@Component
@ConfigurationProperties(prefix = "grafana-auth")
public class GrafanaAuthConfiguration {
@Valid
@NestedConfigurationProperty
private BalancerHttp http = new BalancerHttp();
@Valid
@NestedConfigurationProperty
private BalancerSecurity security = new BalancerSecurity();
@Valid
@NestedConfigurationProperty
private Grafana grafana = new Grafana();
private Integer shutdownQuietPeriod = 5;
public BalancerSecurity getSecurity() {
return security;
}
public BalancerHttp getHttp() {
return http;
}
public Grafana getGrafana() {
return grafana;
}
/**
* Time to wait (in seconds) for connections to finish and to make sure no new
* connections happen before shutting down Netty event loop groups.
*/
public int getShutdownQuietPeriod() {
return this.shutdownQuietPeriod;
}
public void setShutdownQuietPeriod(Integer quietPeriod) {
this.shutdownQuietPeriod = quietPeriod;
}
}
@@ -0,0 +1,7 @@
package timely.grafana.auth.configuration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication(scanBasePackageClasses = { SpringBootstrap.class })
public class SpringBootstrap {
}
@@ -0,0 +1,234 @@
package timely.grafana.auth.netty.http;
import static io.netty.handler.codec.http.HttpHeaderNames.CONTENT_LENGTH;
import java.io.ByteArrayOutputStream;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import com.google.common.collect.Multimap;
import io.netty.buffer.Unpooled;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.handler.codec.http.DefaultFullHttpResponse;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpResponseStatus;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.util.ReferenceCountUtil;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpDelete;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpHead;
import org.apache.http.client.methods.HttpOptions;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.message.BasicHeader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.StreamUtils;
import timely.api.request.timeseries.HttpRequest;
import timely.api.response.TimelyException;
import timely.auth.TimelyPrincipal;
import timely.auth.TimelyUser;
import timely.auth.util.DnUtils;
import timely.auth.util.HttpHeaderUtils;
import timely.balancer.connection.TimelyBalancedHost;
import timely.balancer.connection.http.HttpClientPool;
import timely.grafana.auth.configuration.GrafanaAuthConfiguration;
import timely.grafana.request.GrafanaRequest;
import timely.netty.http.TimelyHttpHandler;
import timely.netty.http.auth.TimelyAuthenticationToken;
public class GrafanaRelayHandler extends SimpleChannelInboundHandler<HttpRequest> implements TimelyHttpHandler {
private static final Logger LOG = LoggerFactory.getLogger(GrafanaRelayHandler.class);
private final HttpClientPool httpClientPool;
private GrafanaAuthConfiguration config;
public GrafanaRelayHandler(GrafanaAuthConfiguration config, HttpClientPool httpClientPool) {
this.config = config;
this.httpClientPool = httpClientPool;
}
@Override
public void channelRead0(ChannelHandlerContext ctx, HttpRequest msg) throws Exception {
CloseableHttpClient client = null;
CloseableHttpResponse relayedResponse = null;
FullHttpRequest request = null;
GrafanaRequest grafanaRequest;
TimelyBalancedHost k = null;
String relayURI = null;
try {
try {
request = msg.getHttpRequest();
String originalURI = request.uri();
originalURI = encodeURI(originalURI);
if (msg instanceof GrafanaRequest) {
grafanaRequest = (GrafanaRequest) msg;
} else {
throw new UnsupportedOperationException("Request not supported");
}
TimelyAuthenticationToken token = grafanaRequest.getToken();
if (token == null) {
throw new IllegalStateException("No token on grafana request");
}
String protocol = config.getGrafana().getProtocol();
String host = config.getGrafana().getHost();
Integer port = config.getGrafana().getPort();
k = TimelyBalancedHost.of(host, -1, port, -1, -1, null);
client = httpClientPool.borrowObject(k);
Multimap<String, String> headers = HttpHeaderUtils.toMultimap(request.headers());
// If incoming user sets these headers, do not relay them
headers.removeAll("x-webauth-user");
headers.removeAll("x-webauth-name");
TimelyPrincipal principal = token.getTimelyPrincipal();
TimelyUser user = principal.getPrimaryUser();
String subjectDn = user.getDn().subjectDN();
String userName = DnUtils.getCommonName(subjectDn);
String loginName = DnUtils.getShortName(subjectDn);
headers.put("x-webauth-user", loginName);
headers.put("x-webauth-name", userName);
List<Header> relayedHeaderList = new ArrayList<>();
for (Map.Entry<String, String> h : headers.entries()) {
if (!h.getKey().equals(CONTENT_LENGTH.toString())) {
relayedHeaderList.add(new BasicHeader(h.getKey(), h.getValue()));
}
}
Header[] headerArray = new Header[relayedHeaderList.size()];
relayedHeaderList.toArray(headerArray);
relayURI = protocol + "://" + host + ":" + port + originalURI;
if (request.method().equals(HttpMethod.HEAD)) {
HttpHead relayedRequest = new HttpHead(relayURI);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.method().equals(HttpMethod.GET)) {
HttpGet relayedRequest = new HttpGet(relayURI);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.method().equals(HttpMethod.POST)) {
HttpPost relayedRequest = new HttpPost(relayURI);
String content = request.content().toString(StandardCharsets.UTF_8);
StringEntity entity = new StringEntity(content);
relayedRequest.setEntity(entity);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.method().equals(HttpMethod.PUT)) {
HttpPut relayedRequest = new HttpPut(relayURI);
String content = request.content().toString(StandardCharsets.UTF_8);
StringEntity entity = new StringEntity(content);
relayedRequest.setEntity(entity);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.method().equals(HttpMethod.DELETE)) {
HttpDelete relayedRequest = new HttpDelete(relayURI);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
} else if (request.method().equals(HttpMethod.OPTIONS)) {
HttpOptions relayedRequest = new HttpOptions(relayURI);
if (headerArray.length > 0) {
relayedRequest.setHeaders(headerArray);
}
relayedResponse = client.execute(relayedRequest);
}
} catch (Exception e) {
String message = e.getMessage();
if (message == null) {
LOG.error("", e);
} else {
LOG.error(message, e);
}
this.sendHttpError(ctx, new TimelyException(HttpResponseStatus.INTERNAL_SERVER_ERROR.code(),
e.getMessage(), e.getLocalizedMessage(), e));
return;
} finally {
if (request != null) {
ReferenceCountUtil.release(request);
}
}
FullHttpResponse response = null;
if (relayedResponse == null) {
this.sendHttpError(ctx, new TimelyException(HttpResponseStatus.METHOD_NOT_ALLOWED.code(),
"Method not allowed", "Method not allowed"));
} else {
LOG.debug("RequestURL:{} ResponseCode:{}", relayURI, relayedResponse.getStatusLine().getStatusCode());
HttpEntity httpEntity = relayedResponse.getEntity();
if (httpEntity == null) {
response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,
HttpResponseStatus.valueOf(relayedResponse.getStatusLine().getStatusCode()));
} else {
ByteArrayOutputStream baos = new MyByteArrayOutputStream();
StreamUtils.copy(httpEntity.getContent(), baos);
response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,
HttpResponseStatus.valueOf(relayedResponse.getStatusLine().getStatusCode()),
Unpooled.copiedBuffer(baos.toByteArray()));
response.headers().set(CONTENT_LENGTH, response.content().readableBytes());
}
for (Header h : relayedResponse.getAllHeaders()) {
response.headers().add(h.getName(), h.getValue());
}
sendResponse(ctx, response);
}
} finally {
if (relayedResponse != null) {
relayedResponse.close();
}
if (client != null && k != null) {
httpClientPool.returnObject(k, client);
} else {
LOG.error("NOT RETURNING CONNECTION! " + msg.getHttpRequest().uri());
}
}
}
static public class MyByteArrayOutputStream extends ByteArrayOutputStream {
public MyByteArrayOutputStream() {
}
public MyByteArrayOutputStream(int size) {
super(size);
}
public int getCount() {
return count;
}
public byte[] getBuf() {
return buf;
}
}
static public String encodeURI(String s) {
try {
s = s.replaceAll("\\{", URLEncoder.encode("{", "UTF-8"));
s = s.replaceAll("\\}", URLEncoder.encode("}", "UTF-8"));
} catch (Exception e) {
}
return s;
}
}
@@ -0,0 +1,133 @@
package timely.grafana.auth.netty.http;
import java.security.cert.X509Certificate;
import java.util.Collection;
import java.util.List;
import com.google.common.collect.Multimap;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.MessageToMessageDecoder;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.QueryStringDecoder;
import io.netty.handler.codec.http.cookie.ServerCookieDecoder;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import timely.api.request.AuthenticatedRequest;
import timely.api.request.timeseries.HttpRequest;
import timely.api.response.StrictTransportResponse;
import timely.api.response.TimelyException;
import timely.auth.AuthCache;
import timely.auth.AuthenticationService;
import timely.auth.SubjectIssuerDNPair;
import timely.auth.TimelyPrincipal;
import timely.auth.util.HttpHeaderUtils;
import timely.configuration.Http;
import timely.configuration.Security;
import timely.grafana.request.GrafanaRequest;
import timely.netty.Constants;
import timely.netty.http.TimelyHttpHandler;
import timely.netty.http.auth.TimelyAuthenticationToken;
public class GrafanaRequestDecoder extends MessageToMessageDecoder<FullHttpRequest> implements TimelyHttpHandler {
private static final Logger LOG = LoggerFactory.getLogger(GrafanaRequestDecoder.class);
private static final String LOG_RECEIVED_REQUEST = "Received HTTP request {}";
private static final String LOG_PARSED_REQUEST = "Parsed request {}";
private final Security security;
private final String nonSecureRedirectAddress;
public GrafanaRequestDecoder(Security security, Http http) {
this.security = security;
this.nonSecureRedirectAddress = http.getRedirectPath();
}
public static String getSessionId(FullHttpRequest msg) {
Multimap<String, String> headers = HttpHeaderUtils.toMultimap(msg.headers());
Collection<String> cookies = headers.get(HttpHeaderNames.COOKIE.toString());
final StringBuilder buf = new StringBuilder();
cookies.forEach(h -> {
ServerCookieDecoder.STRICT.decode(h).forEach(c -> {
if (c.name().equals(Constants.COOKIE_NAME)) {
if (buf.length() == 0) {
buf.append(c.value());
}
}
});
});
if (buf.length() == 0) {
return null;
} else {
return buf.toString();
}
}
@Override
protected void decode(ChannelHandlerContext ctx, FullHttpRequest msg, List<Object> out) throws Exception {
LOG.trace(LOG_RECEIVED_REQUEST, msg);
final String uri = msg.uri();
final QueryStringDecoder decoder = new QueryStringDecoder(uri);
if (decoder.path().equals(nonSecureRedirectAddress)) {
out.add(new StrictTransportResponse());
return;
}
final String sessionId = getSessionId(msg);
LOG.trace("SessionID: " + sessionId);
HttpRequest request;
try {
request = new GrafanaRequest();
Multimap<String, String> headers = HttpHeaderUtils.toMultimap(msg.headers());
((AuthenticatedRequest) request).addHeaders(headers);
X509Certificate clientCert = AuthenticationService.getClientCertificate(ctx);
TimelyAuthenticationToken token;
if (StringUtils.isNotBlank(sessionId)) {
TimelyPrincipal principal;
((AuthenticatedRequest) request).setSessionId(sessionId);
if (AuthCache.containsKey(sessionId)) {
principal = AuthCache.get(sessionId);
} else {
principal = TimelyPrincipal.anonymousPrincipal();
}
SubjectIssuerDNPair dn = principal.getPrimaryUser().getDn();
token = AuthenticationService.getAuthenticationToken(null, dn.subjectDN(), dn.issuerDN());
} else if (clientCert != null) {
token = AuthenticationService.getAuthenticationToken(clientCert, headers);
} else {
SubjectIssuerDNPair dn = TimelyPrincipal.anonymousPrincipal().getPrimaryUser().getDn();
token = AuthenticationService.getAuthenticationToken(null, dn.subjectDN(), dn.issuerDN());
}
((AuthenticatedRequest) request).setToken(token);
request.setHttpRequest(msg.copy());
LOG.trace(LOG_PARSED_REQUEST, request);
request.validate();
out.add(request);
} catch (UnsupportedOperationException | NullPointerException e) {
// Return the original http request to route to the static file
// server
msg.retain();
out.add(msg);
return;
}
try {
try {
AuthenticationService.enforceAccess((AuthenticatedRequest) request);
} catch (TimelyException e) {
if (!security.isAllowAnonymousHttpAccess()) {
throw e;
}
}
} catch (Exception e) {
out.clear();
throw e;
}
}
}
@@ -0,0 +1,32 @@
package timely.grafana.request;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.QueryStringDecoder;
import timely.api.annotation.Http;
import timely.api.request.AuthenticatedRequest;
import timely.api.request.HttpGetRequest;
import timely.api.request.HttpPostRequest;
@Http(path = "/")
public class GrafanaRequest extends AuthenticatedRequest implements HttpGetRequest, HttpPostRequest {
private FullHttpRequest httpRequest = null;
@Override
public HttpGetRequest parseQueryParameters(QueryStringDecoder decoder) throws Exception {
return new GrafanaRequest();
}
@Override
public HttpPostRequest parseBody(String content) throws Exception {
return new GrafanaRequest();
}
public void setHttpRequest(FullHttpRequest httpRequest) {
this.httpRequest = httpRequest;
}
public FullHttpRequest getHttpRequest() {
return httpRequest;
}
}
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<FindBugsFilter>
<Match>
<Class name="timely.grafana.auth.netty.http.GrafanaRelayHandler$MyByteArrayOutputStream" />
<Method name="getBuf" />
<Bug pattern="EI_EXPOSE_REP" />
</Match>
</FindBugsFilter>
+1 -1
View File
@@ -7,7 +7,7 @@
</formats>
<fileSets>
<fileSet>
<directory>timely-app</directory>
<directory>timely-app/dist</directory>
<outputDirectory>timely-app</outputDirectory>
</fileSet>
</fileSets>
+33 -33
View File
@@ -1,33 +1,33 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.1.1-SNAPSHOT</version>
</parent>
<artifactId>timely-app</artifactId>
<packaging>pom</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>assemble</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>timely</groupId>
<artifactId>timely</artifactId>
<version>1.3.3-SNAPSHOT</version>
</parent>
<artifactId>timely-app</artifactId>
<packaging>pom</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>assemble</id>
<goals>
<goal>single</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptors>
<descriptor>assembly.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
-65
View File
@@ -1,65 +0,0 @@
module.exports = function(grunt) {
require('load-grunt-tasks')(grunt);
grunt.loadNpmTasks('grunt-execute');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.initConfig({
clean: ["dist"],
copy: {
src_to_dist: {
cwd: 'src',
expand: true,
src: ['**/*', '!**/*.js', '!**/*.scss'],
dest: 'dist'
},
img_to_dist: {
cwd: 'src',
expand: true,
src: ['img/*'],
dest: 'dist/src/'
},
pluginDef: {
expand: true,
src: ['plugin.json', 'README.md'],
dest: 'dist',
}
},
watch: {
rebuild_all: {
files: ['src/**/*', 'plugin.json', 'README.md'],
tasks: ['default'],
options: {spawn: false}
},
},
babel: {
options: {
sourceMap: true,
presets: ["es2015"],
plugins: ['transform-es2015-modules-systemjs', "transform-es2015-for-of"],
},
dist: {
files: [{
cwd: 'src',
expand: true,
src: ['**/*.js'],
dest: 'dist',
ext:'.js'
}]
},
}
});
grunt.registerTask('default',
['clean',
'copy:src_to_dist',
'copy:pluginDef',
'copy:img_to_dist',
'babel'
]);
};
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+18
View File
@@ -0,0 +1,18 @@
//+build mage
package main
import (
"fmt"
// mage:import
_ "github.com/grafana/grafana-plugin-sdk-go/build"
)
// CheckCheck builds production back-end components.
func CheckCheck() {
fmt.Println("hello! TODO.... (the other commands are imported)")
}
// Default configures the default target.
var Default = CheckCheck
+5 -2
View File
@@ -1,3 +1,6 @@
# Timely
# Timely Data Source Plugin
Timely: a scalable, distributed, **secure** time series database.
Timely is a time series database application that provides secure access to time
series data. Timely is written in Java, uses
[Apache Accumulo](https://accumulo.apache.org/) for backend storage, and has
a configurable length memory cache.
+26
View File
@@ -0,0 +1,26 @@
# Test against the latest version of this Node.js version
environment:
nodejs_version: "10"
# Local NPM Modules
cache:
- node_modules
# Install scripts. (runs after repo cloning)
install:
# Get the latest stable version of Node.js or io.js
- ps: Install-Product node $env:nodejs_version
# install modules
- npm install -g yarn --quiet
- yarn install --pure-lockfile
# Post-install test scripts.
test_script:
# Output useful info for debugging.
- node --version
- npm --version
# Run the build
build_script:
- yarn dev # This will also run prettier!
- yarn build # make sure both scripts work
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+5 -2
View File
@@ -1,3 +1,6 @@
# Timely
# Timely Data Source Plugin
Timely: a scalable, distributed, **secure** time series database.
Timely is a time series database application that provides secure access to time
series data. Timely is written in Java, uses
[Apache Accumulo](https://accumulo.apache.org/) for backend storage, and has
a configurable length memory cache.
@@ -1,10 +0,0 @@
<div class="gf-form-group">
<div class="gf-form">
<span class="gf-form-label width-13">OpenTSDB metrics query</span>
<input type="text" class="gf-form-input" ng-model='ctrl.annotation.target' placeholder="events.eventname"></input>
</div>
<div class="gf-form">
<span class="gf-form-label width-13">Show Global Annotations?</span>
<editor-checkbox text="" model="ctrl.annotation.isGlobal"></editor-checkbox>
</div>
</div>
-52
View File
@@ -1,52 +0,0 @@
<div class="page-header">
<h1>Timely Login</h1>
</div>
<div ng-if="ctrl.datasources.length === 0">
<h4>No Timely datasources are defined.</h4>
<i><a href="datasources/new">Create one</a></i>
</div>
<div ng-repeat="source in ctrl.datasources" ng-init="d=source" ng-controller="AuthsCtrl as actrl">
<div class="gf-box">
<div class="gf-box-header p-x-1 p-y-1">
<h3 class="">{{d.name}}
<span class="pull-right">
<h6><i>https://{{d.jsonData.timelyHost}}:{{d.jsonData.httpsPort}}/login</i></h6>
</span>
</h3>
</div>
<div class="gf-box-body">
<div ng-if="d.jsonData.basicAuths" class="p-b-1">
<div class="gf-form">
<span class="gf-form-label width-7">
User
</span>
<input id="timlely-user" class="gf-form-input max-width-21" type="text" ng-model='actrl.basicAuthUser' placeholder="user" required></input>
</div>
<div class="gf-form">
<span class="gf-form-label width-7">
Password
</span>
<input id="timely-password" class="gf-form-input max-width-21" type="password" ng-model='actrl.basicAuthPassword' placeholder="password" required></input>
</div>
</div>
<div ng-if="actrl.fetching" class="gf-row">
<div class="alert-{{actrl.fetching.status}} alert">
<div class="alert-title">{{actrl.fetching.title}}</div>
</div>
</div>
<div class="gf-form-button-row center-block">
<button type="submit" class="btn btn-success" ng-click="actrl.login(d)">
Login
</button>
</div>
</div>
</div>
</div>
-146
View File
@@ -1,146 +0,0 @@
'use strict';
System.register(['lodash'], function (_export, _context) {
"use strict";
var _, _createClass, AuthsCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_lodash) {
_ = _lodash.default;
}],
execute: function () {
_createClass = function () {
function defineProperties(target, props) {
for (var i = 0; i < props.length; i++) {
var descriptor = props[i];
descriptor.enumerable = descriptor.enumerable || false;
descriptor.configurable = true;
if ("value" in descriptor) descriptor.writable = true;
Object.defineProperty(target, descriptor.key, descriptor);
}
}
return function (Constructor, protoProps, staticProps) {
if (protoProps) defineProperties(Constructor.prototype, protoProps);
if (staticProps) defineProperties(Constructor, staticProps);
return Constructor;
};
}();
_export('AuthsCtrl', AuthsCtrl = function () {
function AuthsCtrl(backendSrv, $rootScope, $timeout) {
_classCallCheck(this, AuthsCtrl);
this.$rootScope = $rootScope;
this.$timeout = $timeout;
this.backendSrv = backendSrv;
}
_createClass(AuthsCtrl, [{
key: 'login',
value: function login(datasource) {
this.fetching = {};
this.fetching.done = false;
this.fetching.show = true;
this.fetching.status = 'warning';
this.fetching.title = 'Fetching auths...';
//this.$rootScope.appEvent('alert-success', ['Authorization Cookie Received', '']);
//this.$rootScope.appEvent('alert-error', ['Authorization Failed', '']);
//this.$rootScope.appEvent('alert-warning', ['Fetching Authorizations', '']);
var uri = this._getURI(datasource.jsonData);
if (datasource.jsonData.basicAuths) {
if (this.basicAuthUser && this.basicAuthPassword) {
var authBlob = { username: this.basicAuthUser, password: this.basicAuthPassword };
this.doPostLogin(uri, authBlob);
} else {
this.$rootScope.appEvent('alert-warning', [datasource.name, 'requires User/Password']);
this.fetching.status = 'warning';
this.fetching.title = 'User/Password required';
return;
}
} else {
this.doGetLogin(uri);
}
}
}, {
key: 'doPostLogin',
value: function doPostLogin(uri, authBlob) {
this.doXHR('POST', uri, authBlob);
}
}, {
key: 'doGetLogin',
value: function doGetLogin(uri) {
this.doXHR('GET', uri);
}
}, {
key: 'doXHR',
value: function doXHR(verb, uri, data) {
if (!data) {
data = "";
}
var r = new XMLHttpRequest();
r.open(verb, uri, true);
r.withCredentials = true;
r.setRequestHeader('Content-Type', 'application/json');
r.onreadystatechange = function () {
this.handleLoginResult(r);
}.bind(this);
r.send(JSON.stringify(data));
}
}, {
key: 'handleLoginResult',
value: function handleLoginResult(response) {
// wait for readyState to be '4 => done'
if (response.readyState != 4) return;
this.fetching.done = true;
switch (response.status) {
case 200:
var cookie = (document.cookie.match(/^(?:.*;)?TSESSIONID=([^;]+)(?:.*)?$/) || [, null])[1];
console.log(cookie);
console.log(document.cookie);
this.fetching.status = 'success';
this.fetching.title = 'Login successful';
this.$rootScope.appEvent('alert-success', ['Success', '']);
break;
default:
this.fetching.status = 'warning';
this.fetching.title = response.statusText;
this.$rootScope.appEvent('alert-warning', [response.status + ': ' + response.statusText, '']);
break;
}
if (r.readyState != 4 || r.status != 200) return;
alert("Success: " + r.getResponseHeader('Set-Cookie'));
}
}, {
key: '_getURI',
value: function _getURI(jsonData) {
return 'https://' + jsonData.timelyHost + ':' + jsonData.httpsPort + '/login';
}
}, {
key: '_showResults',
value: function _showResults() {}
}]);
return AuthsCtrl;
}());
_export('AuthsCtrl', AuthsCtrl);
}
};
});
//# sourceMappingURL=auths.js.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["../../src/components/auths.js"],"names":["_","AuthsCtrl","backendSrv","$rootScope","$timeout","datasource","fetching","done","show","status","title","uri","_getURI","jsonData","basicAuths","basicAuthUser","basicAuthPassword","authBlob","username","password","doPostLogin","appEvent","name","doGetLogin","doXHR","verb","data","r","XMLHttpRequest","open","withCredentials","setRequestHeader","onreadystatechange","handleLoginResult","bind","send","JSON","stringify","response","readyState","cookie","document","match","console","log","statusText","alert","getResponseHeader","timelyHost","httpsPort"],"mappings":";;;;;;;;;;;;;;;AAAOA,O;;;;;;;;;;;;;;;;;;;;;2BAEMC,S;AAEX,2BAAYC,UAAZ,EAAwBC,UAAxB,EAAoCC,QAApC,EAA6C;AAAA;;AAC3C,eAAKD,UAAL,GAAkBA,UAAlB;AACA,eAAKC,QAAL,GAAgBA,QAAhB;AACA,eAAKF,UAAL,GAAkBA,UAAlB;AACD;;;;gCAEKG,U,EAAW;AACf,iBAAKC,QAAL,GAAgB,EAAhB;AACA,iBAAKA,QAAL,CAAcC,IAAd,GAAqB,KAArB;AACA,iBAAKD,QAAL,CAAcE,IAAd,GAAqB,IAArB;AACA,iBAAKF,QAAL,CAAcG,MAAd,GAAuB,SAAvB;AACA,iBAAKH,QAAL,CAAcI,KAAd,GAAsB,mBAAtB;;AAEA;AACA;AACA;;AAEA,gBAAIC,MAAM,KAAKC,OAAL,CAAaP,WAAWQ,QAAxB,CAAV;;AAEA,gBAAGR,WAAWQ,QAAX,CAAoBC,UAAvB,EAAkC;AAChC,kBAAI,KAAKC,aAAL,IAAsB,KAAKC,iBAA/B,EAAiD;AAC/C,oBAAIC,WAAW,EAACC,UAAU,KAAKH,aAAhB,EAA+BI,UAAU,KAAKH,iBAA9C,EAAf;AACA,qBAAKI,WAAL,CAAiBT,GAAjB,EAAsBM,QAAtB;AACD,eAHD,MAGO;AACL,qBAAKd,UAAL,CAAgBkB,QAAhB,CAAyB,eAAzB,EAA0C,CAAChB,WAAWiB,IAAZ,EAAkB,wBAAlB,CAA1C;AACA,qBAAKhB,QAAL,CAAcG,MAAd,GAAuB,SAAvB;AACA,qBAAKH,QAAL,CAAcI,KAAd,GAAsB,wBAAtB;;AAEA;AACD;AACF,aAXD,MAWO;AACL,mBAAKa,UAAL,CAAgBZ,GAAhB;AACD;AAEF;;;sCAEWA,G,EAAKM,Q,EAAS;AACxB,iBAAKO,KAAL,CAAW,MAAX,EAAmBb,GAAnB,EAAwBM,QAAxB;AACD;;;qCAEUN,G,EAAI;AACb,iBAAKa,KAAL,CAAW,KAAX,EAAkBb,GAAlB;AACD;;;gCAEKc,I,EAAMd,G,EAAKe,I,EAAK;AACpB,gBAAI,CAACA,IAAL,EAAU;AACRA,qBAAO,EAAP;AACD;AACD,gBAAIC,IAAI,IAAIC,cAAJ,EAAR;AACAD,cAAEE,IAAF,CAAOJ,IAAP,EAAad,GAAb,EAAkB,IAAlB;AACAgB,cAAEG,eAAF,GAAoB,IAApB;AACAH,cAAEI,gBAAF,CAAmB,cAAnB,EAAmC,kBAAnC;AACAJ,cAAEK,kBAAF,GAAuB,YAAY;AACjC,mBAAKC,iBAAL,CAAuBN,CAAvB;AACD,aAFsB,CAErBO,IAFqB,CAEhB,IAFgB,CAAvB;AAGAP,cAAEQ,IAAF,CAAOC,KAAKC,SAAL,CAAeX,IAAf,CAAP;AACD;;;4CAEiBY,Q,EAAS;AACzB;AACA,gBAAIA,SAASC,UAAT,IAAuB,CAA3B,EAA8B;;AAE9B,iBAAKjC,QAAL,CAAcC,IAAd,GAAqB,IAArB;;AAEA,oBAAO+B,SAAS7B,MAAhB;AACE,mBAAK,GAAL;AACE,oBAAI+B,SAAS,CAACC,SAASD,MAAT,CAAgBE,KAAhB,CAAsB,qCAAtB,KAA8D,GAAE,IAAF,CAA/D,EAAwE,CAAxE,CAAb;AACAC,wBAAQC,GAAR,CAAYJ,MAAZ;AACAG,wBAAQC,GAAR,CAAYH,SAASD,MAArB;;AAEA,qBAAKlC,QAAL,CAAcG,MAAd,GAAuB,SAAvB;AACA,qBAAKH,QAAL,CAAcI,KAAd,GAAsB,kBAAtB;AACA,qBAAKP,UAAL,CAAgBkB,QAAhB,CAAyB,eAAzB,EAA0C,CAAC,SAAD,EAAY,EAAZ,CAA1C;AACA;AACF;AACE,qBAAKf,QAAL,CAAcG,MAAd,GAAuB,SAAvB;AACA,qBAAKH,QAAL,CAAcI,KAAd,GAAsB4B,SAASO,UAA/B;AACA,qBAAK1C,UAAL,CAAgBkB,QAAhB,CAAyB,eAAzB,EAA0C,CAACiB,SAAS7B,MAAT,GAAgB,IAAhB,GAAqB6B,SAASO,UAA/B,EAA0C,EAA1C,CAA1C;AACA;AAdJ;;AAiBA,gBAAIlB,EAAEY,UAAF,IAAgB,CAAhB,IAAqBZ,EAAElB,MAAF,IAAY,GAArC,EAA0C;AAC1CqC,kBAAM,cAAcnB,EAAEoB,iBAAF,CAAoB,YAApB,CAApB;AACD;;;kCAEOlC,Q,EAAS;AACf,mBAAO,aAAWA,SAASmC,UAApB,GAA+B,GAA/B,GAAmCnC,SAASoC,SAA5C,GAAsD,QAA7D;AACD;;;yCAEa,CAEb","file":"auths.js","sourcesContent":["import _ from 'lodash';\n\nexport class AuthsCtrl {\n\n constructor(backendSrv, $rootScope, $timeout){\n this.$rootScope = $rootScope;\n this.$timeout = $timeout;\n this.backendSrv = backendSrv;\n }\n\n login(datasource){\n this.fetching = {};\n this.fetching.done = false;\n this.fetching.show = true;\n this.fetching.status = 'warning';\n this.fetching.title = 'Fetching auths...';\n\n //this.$rootScope.appEvent('alert-success', ['Authorization Cookie Received', '']);\n //this.$rootScope.appEvent('alert-error', ['Authorization Failed', '']);\n //this.$rootScope.appEvent('alert-warning', ['Fetching Authorizations', '']);\n\n var uri = this._getURI(datasource.jsonData);\n\n if(datasource.jsonData.basicAuths){\n if( this.basicAuthUser && this.basicAuthPassword){\n var authBlob = {username: this.basicAuthUser, password: this.basicAuthPassword};\n this.doPostLogin(uri, authBlob);\n } else {\n this.$rootScope.appEvent('alert-warning', [datasource.name, 'requires User/Password']);\n this.fetching.status = 'warning';\n this.fetching.title = 'User/Password required';\n\n return;\n }\n } else {\n this.doGetLogin(uri);\n }\n\n }\n\n doPostLogin(uri, authBlob){\n this.doXHR('POST', uri, authBlob);\n }\n\n doGetLogin(uri){\n this.doXHR('GET', uri)\n }\n\n doXHR(verb, uri, data){\n if( !data){\n data = \"\";\n }\n var r = new XMLHttpRequest();\n r.open(verb, uri, true);\n r.withCredentials = true;\n r.setRequestHeader('Content-Type', 'application/json');\n r.onreadystatechange = function () {\n this.handleLoginResult(r);\n }.bind(this);\n r.send(JSON.stringify(data));\n }\n\n handleLoginResult(response){\n // wait for readyState to be '4 => done'\n if (response.readyState != 4) return;\n\n this.fetching.done = true;\n\n switcLine truncated
-80
View File
@@ -1,80 +0,0 @@
'use strict';
System.register(['lodash', 'app/core/core', 'plugins/timely-app/components/auths'], function (_export, _context) {
"use strict";
var _, coreModule, appEvents, AuthsCtrl, _createClass, AuthsPageCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_lodash) {
_ = _lodash.default;
}, function (_appCoreCore) {
coreModule = _appCoreCore.coreModule;
appEvents = _appCoreCore.appEvents;
}, function (_pluginsTimelyAppComponentsAuths) {
AuthsCtrl = _pluginsTimelyAppComponentsAuths.AuthsCtrl;
}],
execute: function () {
_createClass = function () {
function defineProperties(target, props) {
for (var i = 0; i < props.length; i++) {
var descriptor = props[i];
descriptor.enumerable = descriptor.enumerable || false;
descriptor.configurable = true;
if ("value" in descriptor) descriptor.writable = true;
Object.defineProperty(target, descriptor.key, descriptor);
}
}
return function (Constructor, protoProps, staticProps) {
if (protoProps) defineProperties(Constructor.prototype, protoProps);
if (staticProps) defineProperties(Constructor, staticProps);
return Constructor;
};
}();
_export('AuthsPageCtrl', AuthsPageCtrl = function () {
function AuthsPageCtrl(backendSrv, datasourceSrv) {
_classCallCheck(this, AuthsPageCtrl);
this.datasourceSrv = datasourceSrv;
this.datasources = this.getTimelyDatasources();
coreModule.controller('AuthsCtrl', AuthsCtrl);
}
_createClass(AuthsPageCtrl, [{
key: 'getTimelyDatasources',
value: function getTimelyDatasources() {
var allSources = this.datasourceSrv.getAll();
var sources = _.map(allSources, function (value, key, col) {
var ret = {};
ret.meta = {};
ret.meta.type = col[key].meta.type;
ret.meta.id = col[key].meta.id;
ret.name = key;
ret.jsonData = col[key].jsonData;
return ret;
});
return _.filter(sources, function (d) {
return d.meta.type === 'datasource' && d.meta.id === 'timely-datasource';
});
}
}]);
return AuthsPageCtrl;
}());
_export('AuthsPageCtrl', AuthsPageCtrl);
AuthsPageCtrl.templateUrl = 'components/auths.html';
}
};
});
//# sourceMappingURL=authspage.js.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["../../src/components/authspage.js"],"names":["_","coreModule","appEvents","AuthsCtrl","AuthsPageCtrl","backendSrv","datasourceSrv","datasources","getTimelyDatasources","controller","allSources","getAll","sources","map","value","key","col","ret","meta","type","id","name","jsonData","filter","d","templateUrl"],"mappings":";;;;;;;;;;;;;;;AACOA,O;;AACCC,gB,gBAAAA,U;AAAYC,e,gBAAAA,S;;AACZC,e,oCAAAA,S;;;;;;;;;;;;;;;;;;;;;+BAEKC,a;AAEX,+BAAYC,UAAZ,EAAwBC,aAAxB,EAAsC;AAAA;;AACpC,eAAKA,aAAL,GAAqBA,aAArB;AACA,eAAKC,WAAL,GAAmB,KAAKC,oBAAL,EAAnB;AACAP,qBAAWQ,UAAX,CAAsB,WAAtB,EAAmCN,SAAnC;AACD;;;;iDAEqB;AACpB,gBAAIO,aAAa,KAAKJ,aAAL,CAAmBK,MAAnB,EAAjB;;AAEA,gBAAIC,UAAUZ,EAAEa,GAAF,CAAMH,UAAN,EAAkB,UAASI,KAAT,EAAgBC,GAAhB,EAAqBC,GAArB,EAAyB;AACvD,kBAAIC,MAAM,EAAV;AACAA,kBAAIC,IAAJ,GAAW,EAAX;AACAD,kBAAIC,IAAJ,CAASC,IAAT,GAAgBH,IAAID,GAAJ,EAASG,IAAT,CAAcC,IAA9B;AACAF,kBAAIC,IAAJ,CAASE,EAAT,GAAcJ,IAAID,GAAJ,EAASG,IAAT,CAAcE,EAA5B;AACAH,kBAAII,IAAJ,GAAWN,GAAX;AACAE,kBAAIK,QAAJ,GAAeN,IAAID,GAAJ,EAASO,QAAxB;AACA,qBAAOL,GAAP;AACD,aARa,CAAd;AASA,mBAAOjB,EAAEuB,MAAF,CAASX,OAAT,EAAkB,UAASY,CAAT,EAAW;AAClC,qBAAQA,EAAEN,IAAF,CAAOC,IAAP,KAAgB,YAAhB,IAAgCK,EAAEN,IAAF,CAAOE,EAAP,KAAc,mBAAtD;AACD,aAFM,CAAP;AAGD;;;;;;;;AAGHhB,oBAAcqB,WAAd,GAA4B,uBAA5B","file":"authspage.js","sourcesContent":["\nimport _ from 'lodash';\nimport {coreModule, appEvents} from 'app/core/core';\nimport {AuthsCtrl} from 'plugins/timely-app/components/auths';\n\nexport class AuthsPageCtrl {\n\n constructor(backendSrv, datasourceSrv){\n this.datasourceSrv = datasourceSrv;\n this.datasources = this.getTimelyDatasources();\n coreModule.controller('AuthsCtrl', AuthsCtrl);\n }\n\n getTimelyDatasources(){\n var allSources = this.datasourceSrv.getAll();\n\n var sources = _.map(allSources, function(value, key, col){\n var ret = {};\n ret.meta = {};\n ret.meta.type = col[key].meta.type;\n ret.meta.id = col[key].meta.id;\n ret.name = key;\n ret.jsonData = col[key].jsonData;\n return ret;\n });\n return _.filter(sources, function(d){\n return (d.meta.type === 'datasource' && d.meta.id === 'timely-datasource');\n });\n }\n\n}\nAuthsPageCtrl.templateUrl = 'components/auths.html';\n"]}
-10
View File
@@ -1,10 +0,0 @@
<h3 class="page-heading">Timely App Configuration</h3>
<div class="gf-form-group">
<div class="gf-form-inline">
<div class="gf-form">
<span class="gf-form-label">Currently, all configuration is done in the Timely datasource.</span>
</div>
</div>
</div>
-27
View File
@@ -1,27 +0,0 @@
'use strict';
System.register([], function (_export, _context) {
"use strict";
var TimelyAppConfigCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [],
execute: function () {
_export('TimelyAppConfigCtrl', TimelyAppConfigCtrl = function TimelyAppConfigCtrl() {
_classCallCheck(this, TimelyAppConfigCtrl);
});
_export('TimelyAppConfigCtrl', TimelyAppConfigCtrl);
TimelyAppConfigCtrl.templateUrl = 'components/config.html';
}
};
});
//# sourceMappingURL=config.js.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["../../src/components/config.js"],"names":["TimelyAppConfigCtrl","templateUrl"],"mappings":";;;;;;;;;;;;;;;;qCACaA,mB,GAEX,+BAAc;AAAA;AAEb,O;;;;AAGHA,0BAAoBC,WAApB,GAAkC,wBAAlC","file":"config.js","sourcesContent":["\nexport class TimelyAppConfigCtrl {\n\n constructor() {\n \n }\n\n}\nTimelyAppConfigCtrl.templateUrl = 'components/config.html';\n"]}
-53
View File
@@ -1,53 +0,0 @@
<div class="page-header">
<h1>Metrics Browser</h1>
</div>
<div ng-hide="ctrl.hasDatasource" >
<h4>No Timely datasources are defined.</h4>
<i><a href="datasources/new">Create one</a></i>
</div>
<div class="gf-box" ng-show="ctrl.hasDatasource">
<div class="gf-box-header">
<div class="gf-form-inline">
<div class="gf-form max-width-30">
<span class="gf-label width-7 p-y-1 p-x-1">Datasource</span>
<div class="gf-form-select-wrapper gf-form-select-wrapper--has-help-icon max-width-24">
<select class="gf-form-input"
ng-model="ctrl.source"
ng-options="d.name for d in ctrl.datasources"
ng-change=ctrl.updateDatasource()></select>
<info-popover mode="right-absolute">
Select a Datasource<br>
</info-popover>
</div>
</div>
</div>
</div>
<div class="gf-box-body">
<div class="grafana-options-table">
<div class="gf-form width-20 pull-right">
<input type="text" class="gf-form-input" placeholder="filter" ng-model="searchText">
</input>
</div>
<table>
<tr>
<th>Metric</th>
<th>Tags</th>
</tr>
<tr ng-repeat="metric in ctrl.metrics | filter:searchText">
<td>{{metric.metric}}</td>
<td class="p-x-1" ng-repeat="tag in metric.tags">
{{tag.name}} = {{tag.values}}
</td>
</tr>
</table>
</div>
</div>
</div>
-124
View File
@@ -1,124 +0,0 @@
'use strict';
System.register(['lodash', 'plugins/timely-app/components/authspage'], function (_export, _context) {
"use strict";
var _, AuthsPageCtrl, _createClass, MetricsPageCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_lodash) {
_ = _lodash.default;
}, function (_pluginsTimelyAppComponentsAuthspage) {
AuthsPageCtrl = _pluginsTimelyAppComponentsAuthspage.AuthsPageCtrl;
}],
execute: function () {
_createClass = function () {
function defineProperties(target, props) {
for (var i = 0; i < props.length; i++) {
var descriptor = props[i];
descriptor.enumerable = descriptor.enumerable || false;
descriptor.configurable = true;
if ("value" in descriptor) descriptor.writable = true;
Object.defineProperty(target, descriptor.key, descriptor);
}
}
return function (Constructor, protoProps, staticProps) {
if (protoProps) defineProperties(Constructor.prototype, protoProps);
if (staticProps) defineProperties(Constructor, staticProps);
return Constructor;
};
}();
_export('MetricsPageCtrl', MetricsPageCtrl = function () {
function MetricsPageCtrl(backendSrv, datasourceSrv) {
_classCallCheck(this, MetricsPageCtrl);
this.hasDatasource = false;
this.backendSrv = backendSrv;
var apc = new AuthsPageCtrl(backendSrv, datasourceSrv);
this.datasources = apc.getTimelyDatasources();
if (this.datasources.length > 0) {
this.source = this.datasources[0];
this.updateDatasource();
}
}
_createClass(MetricsPageCtrl, [{
key: 'updateDatasource',
value: function updateDatasource() {
var _this = this;
this._clearMetrics();
var uri = this._getURI(this.source.jsonData);
this.backendSrv.request({ method: 'GET',
url: uri,
withCredentials: true,
params: "" }).then(function (response) {
_this.metrics = _this._transformMetrics(response.metrics);
_this.hasDatasource = true;
});
}
}, {
key: '_clearMetrics',
value: function _clearMetrics() {
this.metrics = [];
}
}, {
key: '_transformMetrics',
value: function _transformMetrics(metrics) {
var ret = [];
_.each(metrics, function (value, idx) {
var metric = {};
metric.metric = value.metric;
metric.tags = [];
_.map(value.tags, function (tag, key) {
// result || (result = {});
// result['name'] = tag.key;
// result['values'] || (result['values'] = []);
// result['values'].push(tag.value);
var gIdx = _.findIndex(metric.tags, { 'name': tag.key });
var grouped;
if (gIdx == -1) {
grouped = { name: '', values: [] };
} else {
grouped = metric.tags[gIdx];
}
grouped.values.push(tag.value);
grouped.values = _.sortBy(_.uniq(grouped.values));
grouped.name = tag.key;
if (gIdx > -1) {
metric.tags[gIdx] = grouped;
} else {
metric.tags.push(grouped);
}
}.bind(metric));
metric.tags = _.sortBy(metric.tags, 'name');
ret.push(metric);
}.bind(ret));
return _.sortBy(ret, 'metric');
}
}, {
key: '_getURI',
value: function _getURI(jsonData) {
return 'https://' + jsonData.timelyHost + ':' + jsonData.httpsPort + '/api/metrics';
}
}]);
return MetricsPageCtrl;
}());
_export('MetricsPageCtrl', MetricsPageCtrl);
MetricsPageCtrl.templateUrl = 'components/metrics.html';
}
};
});
//# sourceMappingURL=metrics.js.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["../../src/components/metrics.js"],"names":["_","AuthsPageCtrl","MetricsPageCtrl","backendSrv","datasourceSrv","hasDatasource","apc","datasources","getTimelyDatasources","length","source","updateDatasource","_clearMetrics","uri","_getURI","jsonData","request","method","url","withCredentials","params","then","metrics","_transformMetrics","response","ret","each","value","idx","metric","tags","map","tag","key","gIdx","findIndex","grouped","name","values","push","sortBy","uniq","bind","timelyHost","httpsPort","templateUrl"],"mappings":";;;;;;;;;;;;;;;AACOA,O;;AACCC,mB,wCAAAA,a;;;;;;;;;;;;;;;;;;;;;iCAEKC,e;AAEX,iCAAYC,UAAZ,EAAwBC,aAAxB,EAAsC;AAAA;;AACpC,eAAKC,aAAL,GAAqB,KAArB;AACA,eAAKF,UAAL,GAAkBA,UAAlB;AACA,cAAIG,MAAM,IAAIL,aAAJ,CAAkBE,UAAlB,EAA8BC,aAA9B,CAAV;AACA,eAAKG,WAAL,GAAmBD,IAAIE,oBAAJ,EAAnB;AACA,cAAI,KAAKD,WAAL,CAAiBE,MAAjB,GAA0B,CAA9B,EAAgC;AAC9B,iBAAKC,MAAL,GAAc,KAAKH,WAAL,CAAiB,CAAjB,CAAd;AACA,iBAAKI,gBAAL;AACD;AACF;;;;6CAEiB;AAAA;;AAChB,iBAAKC,aAAL;AACA,gBAAIC,MAAM,KAAKC,OAAL,CAAa,KAAKJ,MAAL,CAAYK,QAAzB,CAAV;AACA,iBAAKZ,UAAL,CAAgBa,OAAhB,CACE,EAAEC,QAAQ,KAAV;AACEC,mBAAKL,GADP;AAEEM,+BAAiB,IAFnB;AAGEC,sBAAQ,EAHV,EADF,EAKEC,IALF,CAME,oBAAY;AACV,oBAAKC,OAAL,GAAe,MAAKC,iBAAL,CAAuBC,SAASF,OAAhC,CAAf;AACA,oBAAKjB,aAAL,GAAqB,IAArB;AACD,aATH;AAUD;;;0CAEc;AACb,iBAAKiB,OAAL,GAAe,EAAf;AACD;;;4CAEiBA,O,EAAQ;AACxB,gBAAIG,MAAM,EAAV;AACAzB,cAAE0B,IAAF,CAAOJ,OAAP,EAAgB,UAASK,KAAT,EAAgBC,GAAhB,EAAoB;AAClC,kBAAIC,SAAS,EAAb;AACAA,qBAAOA,MAAP,GAAgBF,MAAME,MAAtB;AACAA,qBAAOC,IAAP,GAAc,EAAd;AACA9B,gBAAE+B,GAAF,CAAMJ,MAAMG,IAAZ,EAAkB,UAASE,GAAT,EAAcC,GAAd,EAAkB;AAC1C;AACA;AACA;AACA;AACQ,oBAAIC,OAAOlC,EAAEmC,SAAF,CAAYN,OAAOC,IAAnB,EAAyB,EAAC,QAAQE,IAAIC,GAAb,EAAzB,CAAX;AACA,oBAAIG,OAAJ;AACA,oBAAIF,QAAQ,CAAC,CAAb,EAAgB;AACdE,4BAAU,EAACC,MAAM,EAAP,EAAWC,QAAQ,EAAnB,EAAV;AACD,iBAFD,MAEO;AACLF,4BAAUP,OAAOC,IAAP,CAAYI,IAAZ,CAAV;AACD;AACDE,wBAAQE,MAAR,CAAeC,IAAf,CAAoBP,IAAIL,KAAxB;AACAS,wBAAQE,MAAR,GAAiBtC,EAAEwC,MAAF,CAASxC,EAAEyC,IAAF,CAAOL,QAAQE,MAAf,CAAT,CAAjB;AACAF,wBAAQC,IAAR,GAAeL,IAAIC,GAAnB;AACA,oBAAIC,OAAO,CAAC,CAAZ,EAAe;AACbL,yBAAOC,IAAP,CAAYI,IAAZ,IAAoBE,OAApB;AACD,iBAFD,MAEO;AACLP,yBAAOC,IAAP,CAAYS,IAAZ,CAAiBH,OAAjB;AACD;AACF,eApBiB,CAoBhBM,IApBgB,CAoBXb,MApBW,CAAlB;AAqBAA,qBAAOC,IAAP,GAAc9B,EAAEwC,MAAF,CAASX,OAAOC,IAAhB,EAAsB,MAAtB,CAAd;AACAL,kBAAIc,IAAJ,CAASV,MAAT;AACD,aA3Be,CA2Bda,IA3Bc,CA2BTjB,GA3BS,CAAhB;;AA6BA,mBAAOzB,EAAEwC,MAAF,CAASf,GAAT,EAAc,QAAd,CAAP;AACD;;;kCAEOV,Q,EAAS;AACf,mBAAO,aAAWA,SAAS4B,UAApB,GAA+B,GAA/B,GAAmC5B,SAAS6B,SAA5C,GAAsD,cAA7D;AACD;;;;;;;;AAGH1C,sBAAgB2C,WAAhB,GAA8B,yBAA9B","file":"metrics.js","sourcesContent":["\nimport _ from 'lodash';\nimport {AuthsPageCtrl} from 'plugins/timely-app/components/authspage';\n\nexport class MetricsPageCtrl {\n\n constructor(backendSrv, datasourceSrv){\n this.hasDatasource = false;\n this.backendSrv = backendSrv;\n var apc = new AuthsPageCtrl(backendSrv, datasourceSrv);\n this.datasources = apc.getTimelyDatasources();\n if( this.datasources.length > 0){\n this.source = this.datasources[0];\n this.updateDatasource();\n }\n }\n\n updateDatasource(){\n this._clearMetrics();\n var uri = this._getURI(this.source.jsonData);\n this.backendSrv.request(\n { method: 'GET',\n url: uri,\n withCredentials: true,\n params: \"\" }\n ).then(\n response => {\n this.metrics = this._transformMetrics(response.metrics);\n this.hasDatasource = true;\n });\n }\n\n _clearMetrics(){\n this.metrics = [];\n }\n\n _transformMetrics(metrics){\n var ret = [];\n _.each(metrics, function(value, idx){\n var metric = {};\n metric.metric = value.metric;\n metric.tags = [];\n _.map(value.tags, function(tag, key){\n// result || (result = {});\n// result['name'] = tag.key;\n// result['values'] || (result['values'] = []);\n// result['values'].push(tag.value);\n var gIdx = _.findIndex(metric.tags, {'name': tag.key});\n var grouped;\n if( gIdx == -1 ){\n grouped = {name: '', values: []};\n } else {\n grouped = metric.tags[gIdx];\n }\n grouped.values.push(tag.value);\n grouped.values = _.sortBy(_.uniq(grouped.values))\n grouped.name = tag.key;\n if( gIdx > -1 ){\n metric.tags[gIdx] = grouped;\n } else {\n metric.tags.push(grouped);\n }\n }.bind(metric));\n metric.tags = _.sortBy(metric.tags, 'name')\n ret.push(metric);\n }.bind(ret));\n\n return _.sortBy(ret, 'metric');\n }\n\n _getURI(jsonData){\n return 'https://'+jsonData.timelyHost+':'+jsonData.httpsPort+'/api/metrics'\n }\n\n}\nMetricsPageCtrl.templateUrl = 'components/metrics.html';\n"]}
-257
View File
@@ -1,257 +0,0 @@
<query-editor-row query-ctrl="ctrl" can-collapse="false">
<div class="gf-form-inline">
<div class="gf-form max-width-25">
<label class="gf-form-label query-keyword width-8">
Metric
<label class="gf-form-label" bs-tooltip="ctrl.errors.metric" style="color: rgb(229, 189, 28)" ng-show="ctrl.errors.metric">
<i class="fa fa-warning"></i>
</label>
</label>
<input type="text" class="gf-form-input" ng-model="ctrl.target.metric"
spellcheck='false' bs-typeahead="ctrl.suggestMetrics" placeholder="metric name" data-min-length=0 data-items=100
ng-blur="ctrl.targetBlur()">
</input>
</div>
<div class="gf-form">
<label class="gf-form-label query-keyword">
Aggregator
<a bs-tooltip="ctrl.errors.aggregator" style="color: rgb(229, 189, 28)" ng-show="ctrl.errors.aggregator">
<i class="fa fa-warning"></i>
</a>
</label>
<div class="gf-form-select-wrapper max-width-15">
<select ng-model="ctrl.target.aggregator" class="gf-form-input"
ng-options="agg for agg in ctrl.aggregators"
ng-change="ctrl.targetBlur()">
</select>
</div>
</div>
<div class="gf-form max-width-20">
<label class="gf-form-label query-keyword width-6">
Alias:
<info-popover mode="right-normal">
Use patterns like $tag_tagname to replace part of the alias for a tag value
</info-popover>
</label>
<input type="text" class="gf-form-input"
ng-model="ctrl.target.alias"
spellcheck='false'
placeholder="series alias"
data-min-length=0 data-items=100
ng-blur="ctrl.targetBlur()"></input>
</div>
<div class="gf-form gf-form--grow">
<div class="gf-form-label gf-form-label--grow"></div>
</div>
</div>
<div class="gf-form-inline">
<div class="gf-form max-width-25">
<label class="gf-form-label query-keyword width-8">Down sample</label>
<input type="text" class="gf-form-input"
ng-model="ctrl.target.downsampleInterval"
ng-model-onblur
ng-change="ctrl.targetBlur()"
placeholder="interval"></input>
<info-popover mode="right-absolute">
blank for auto, or for example <code>1m</code>
</info-popover>
</div>
<div class="gf-form">
<label class="gf-form-label query-keyword">Aggregator</label>
<div class="gf-form-select-wrapper">
<select ng-model="ctrl.target.downsampleAggregator" class="gf-form-input"
ng-options="agg for agg in ctrl.aggregators"
ng-change="ctrl.targetBlur()">
</select>
</div>
</div>
<div class="gf-form" ng-if="ctrl.tsdbVersion >= 2">
<label class="gf-form-label query-keyword width-6">Fill</label>
<div class="gf-form-select-wrapper">
<select ng-model="ctrl.target.downsampleFillPolicy" class="gf-form-input"
ng-options="agg for agg in ctrl.fillPolicies"
ng-change="ctrl.targetBlur()">
</select>
</div>
</div>
<gf-form-switch class="gf-form"
label="Disable downsampling"
label-class="query-keyword"
checked="ctrl.target.disableDownsampling"
on-change="ctrl.targetBlur()">
</gf-form-switch>
<div class="gf-form gf-form--grow">
<div class="gf-form-label gf-form-label--grow"></div>
</div>
</div>
<div class="gf-form-inline" ng-if="ctrl.tsdbVersion >= 2">
<div class="gf-form">
<label class="gf-form-label query-keyword width-8">
Filters
<info-popover mode="right-normal">
Filters does not work with tags, either of the two will work but not both.
</info-popover>
</label>
<div ng-repeat="fil in ctrl.target.filters track by $index" class="gf-form-label">
{{fil.tagk}}&nbsp;=&nbsp;{{fil.type}}&#40;{{fil.filter}}&#41;&nbsp;&#44&nbsp;groupBy&nbsp;=&nbsp;{{fil.groupBy}}
<a ng-click="ctrl.editFilter(fil, $index)">
<i class="fa fa-pencil"></i>
</a>
<a ng-click="ctrl.removeFilter($index)">
<i class="fa fa-remove"></i>
</a>
</div>
<label class="gf-form-label query-keyword" ng-hide="ctrl.addFilterMode">
<a ng-click="ctrl.addFilter()">
<i class="fa fa-plus"></i>
</a>
</label>
</div>
<div class="gf-form-inline" ng-show="ctrl.addFilterMode">
<div class="gf-form">
<input type="text" class="gf-form-input" spellcheck='false'
bs-typeahead="ctrl.suggestTagKeys" data-min-length=0 data-items=100
ng-model="ctrl.target.currentFilterKey" placeholder="key">
</input>
</div>
<div class="gf-form">
<label class="gf-form-label query-keyword">Type</label>
<div class="gf-form-select-wrapper">
<select ng-model="ctrl.target.currentFilterType" class="gf-form-input" ng-options="filType for filType in ctrl.filterTypes">
</select>
</div>
</div>
<div class="gf-form">
<input type="text" class="gf-form-input" spellcheck='false' bs-typeahead="ctrl.suggestTagValues" data-min-length=0 data-items=100 ng-model="ctrl.target.currentFilterValue" placeholder="filter">
</input>
</div>
<gf-form-switch class="gf-form"
label="Group by"
label-class="query-keyword"
checked="ctrl.target.currentFilterGroupBy"
on-change="ctrl.targetBlur()">
</gf-form-switch>
<div class="gf-form" ng-show="ctrl.addFilterMode">
<label class="gf-form-label" ng-show="ctrl.errors.filters">
<a bs-tooltip="ctrl.errors.filters" style="color: rgb(229, 189, 28)" >
<i class="fa fa-warning"></i>
</a>
</label>
<label class="gf-form-label">
<a ng-click="ctrl.addFilter()" ng-hide="ctrl.errors.filters">add filter</a>
<a ng-click="ctrl.closeAddFilterMode()">
<i class="fa fa-remove"></i>
</a>
</label>
</div>
</div>
<div class="gf-form gf-form--grow">
<div class="gf-form-label gf-form-label--grow"></div>
</div>
</div>
<div class="gf-form-inline">
<div class="gf-form">
<label class="gf-form-label query-keyword width-8">
Tags
<info-popover mode="right-normal" ng-if="ctrl.tsdbVersion >= 2">
Please use filters, tags are deprecated in opentsdb 2.2
</info-popover>
</label>
</div>
<div class="gf-form" ng-repeat="(key, value) in ctrl.target.tags track by $index" class="gf-form">
<label class="gf-form-label">
{{key}}&nbsp;=&nbsp;{{value}}
<a ng-click="ctrl.editTag(key, value)">
<i class="fa fa-pencil"></i>
</a>
<a ng-click="ctrl.removeTag(key)">
<i class="fa fa-remove"></i>
</a>
</label>
</div>
<div class="gf-form" ng-hide="ctrl.addTagMode">
<label class="gf-form-label query-keyword">
<a ng-click="ctrl.addTag()"><i class="fa fa-plus"></i></a>
</label>
</div>
<div class="gf-form" ng-show="ctrl.addTagMode">
<input type="text"
class="gf-form-input" spellcheck='false'
bs-typeahead="ctrl.suggestTagKeys" data-min-length=0 data-items=100
ng-model="ctrl.target.currentTagKey" placeholder="key">
</input>
<input type="text" class="gf-form-input"
spellcheck='false' bs-typeahead="ctrl.suggestTagValues"
data-min-length=0 data-items=100 ng-model="ctrl.target.currentTagValue" placeholder="value">
</input>
<label class="gf-form-label" ng-show="ctrl.errors.tags">
<a bs-tooltip="ctrl.errors.tags" style="color: rgb(229, 189, 28)" >
<i class="fa fa-warning"></i>
</a>
</label>
<label class="gf-form-label" >
<a ng-click="ctrl.addTag()" ng-hide="ctrl.errors.tags">add tag</a>
<a ng-click="ctrl.closeAddTagMode()"><i class="fa fa-remove"></i></a>
</label>
</div>
<div class="gf-form gf-form--grow">
<div class="gf-form-label gf-form-label--grow"></div>
</div>
</div>
<div class="gf-form-inline">
<gf-form-switch class="gf-form" label="Rate" label-class="width-8 query-keyword" checked="ctrl.target.shouldComputeRate" on-change="ctrl.targetBlur()">
</gf-form-switch>
<gf-form-switch ng-hide="!ctrl.target.shouldComputeRate"
class="gf-form" label="Counter" label-class="query-keyword" checked="ctrl.target.isCounter" on-change="ctrl.targetBlur()">
</gf-form-switch>
<div class="gf-form" ng-hide="!ctrl.target.isCounter || !ctrl.target.shouldComputeRate">
<label class="gf-form-label query-keyword">Counter Max</label>
<input type="text" class="gf-form-input"
ng-disabled="!ctrl.target.shouldComputeRate"
ng-model="ctrl.target.counterMax" spellcheck='false'
placeholder="max value" ng-model-onblur
ng-blur="ctrl.targetBlur()">
</input>
<label class="gf-form-label query-keyword">Reset Value</label>
<input type="text" class="tight-form-input input-small"
ng-disabled="!ctrl.target.shouldComputeRate"
ng-model="ctrl.target.counterResetValue" spellcheck='false'
placeholder="reset value" ng-model-onblur
ng-blur="ctrl.targetBlur()">
</input>
</div>
<div class="gf-form gf-form--grow">
<div class="gf-form-label gf-form-label--grow"></div>
</div>
</div>
</query-editor-row>
View File
Whitespace-only changes.
View File
Whitespace-only changes.
-3
View File
@@ -1,3 +0,0 @@
.timely-app-heading {
text-align: center;
}
+101
View File
@@ -0,0 +1,101 @@
define(function() { return /******/ (function(modules) { // webpackBootstrap
/******/ // The module cache
/******/ var installedModules = {};
/******/
/******/ // The require function
/******/ function __webpack_require__(moduleId) {
/******/
/******/ // Check if module is in cache
/******/ if(installedModules[moduleId]) {
/******/ return installedModules[moduleId].exports;
/******/ }
/******/ // Create a new module (and put it into the cache)
/******/ var module = installedModules[moduleId] = {
/******/ i: moduleId,
/******/ l: false,
/******/ exports: {}
/******/ };
/******/
/******/ // Execute the module function
/******/ modules[moduleId].call(module.exports, module, module.exports, __webpack_require__);
/******/
/******/ // Flag the module as loaded
/******/ module.l = true;
/******/
/******/ // Return the exports of the module
/******/ return module.exports;
/******/ }
/******/
/******/
/******/ // expose the modules object (__webpack_modules__)
/******/ __webpack_require__.m = modules;
/******/
/******/ // expose the module cache
/******/ __webpack_require__.c = installedModules;
/******/
/******/ // define getter function for harmony exports
/******/ __webpack_require__.d = function(exports, name, getter) {
/******/ if(!__webpack_require__.o(exports, name)) {
/******/ Object.defineProperty(exports, name, { enumerable: true, get: getter });
/******/ }
/******/ };
/******/
/******/ // define __esModule on exports
/******/ __webpack_require__.r = function(exports) {
/******/ if(typeof Symbol !== 'undefined' && Symbol.toStringTag) {
/******/ Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' });
/******/ }
/******/ Object.defineProperty(exports, '__esModule', { value: true });
/******/ };
/******/
/******/ // create a fake namespace object
/******/ // mode & 1: value is a module id, require it
/******/ // mode & 2: merge all properties of value into the ns
/******/ // mode & 4: return value when already ns object
/******/ // mode & 8|1: behave like require
/******/ __webpack_require__.t = function(value, mode) {
/******/ if(mode & 1) value = __webpack_require__(value);
/******/ if(mode & 8) return value;
/******/ if((mode & 4) && typeof value === 'object' && value && value.__esModule) return value;
/******/ var ns = Object.create(null);
/******/ __webpack_require__.r(ns);
/******/ Object.defineProperty(ns, 'default', { enumerable: true, value: value });
/******/ if(mode & 2 && typeof value != 'string') for(var key in value) __webpack_require__.d(ns, key, function(key) { return value[key]; }.bind(null, key));
/******/ return ns;
/******/ };
/******/
/******/ // getDefaultExport function for compatibility with non-harmony modules
/******/ __webpack_require__.n = function(module) {
/******/ var getter = module && module.__esModule ?
/******/ function getDefault() { return module['default']; } :
/******/ function getModuleExports() { return module; };
/******/ __webpack_require__.d(getter, 'a', getter);
/******/ return getter;
/******/ };
/******/
/******/ // Object.prototype.hasOwnProperty.call
/******/ __webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); };
/******/
/******/ // __webpack_public_path__
/******/ __webpack_require__.p = "/";
/******/
/******/
/******/ // Load entry module and return exports
/******/ return __webpack_require__(__webpack_require__.s = "./styles/dark.css");
/******/ })
/************************************************************************/
/******/ ({
/***/ "./styles/dark.css":
/*!*************************!*\
!*** ./styles/dark.css ***!
\*************************/
/*! no static exports found */
/***/ (function(module, exports, __webpack_require__) {
// extracted by mini-css-extract-plugin
/***/ })
/******/ })});;
//# sourceMappingURL=dark.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"sources":["webpack:///webpack/bootstrap","webpack:///./styles/dark.css"],"names":[],"mappings":";QAAA;QACA;;QAEA;QACA;;QAEA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;;QAEA;QACA;;QAEA;QACA;;QAEA;QACA;QACA;;;QAGA;QACA;;QAEA;QACA;;QAEA;QACA;QACA;QACA,0CAA0C,gCAAgC;QAC1E;QACA;;QAEA;QACA;QACA;QACA,wDAAwD,kBAAkB;QAC1E;QACA,iDAAiD,cAAc;QAC/D;;QAEA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA;QACA,yCAAyC,iCAAiC;QAC1E,gHAAgH,mBAAmB,EAAE;QACrI;QACA;;QAEA;QACA;QACA;QACA,2BAA2B,0BAA0B,EAAE;QACvD,iCAAiC,eAAe;QAChD;QACA;QACA;;QAEA;QACA,sDAAsD,+DAA+D;;QAErH;QACA;;;QAGA;QACA;;;;;;;;;;;;AClFA,uC","file":"dark.js","sourcesContent":[" \t// The module cache\n \tvar installedModules = {};\n\n \t// The require function\n \tfunction __webpack_require__(moduleId) {\n\n \t\t// Check if module is in cache\n \t\tif(installedModules[moduleId]) {\n \t\t\treturn installedModules[moduleId].exports;\n \t\t}\n \t\t// Create a new module (and put it into the cache)\n \t\tvar module = installedModules[moduleId] = {\n \t\t\ti: moduleId,\n \t\t\tl: false,\n \t\t\texports: {}\n \t\t};\n\n \t\t// Execute the module function\n \t\tmodules[moduleId].call(module.exports, module, module.exports, __webpack_require__);\n\n \t\t// Flag the module as loaded\n \t\tmodule.l = true;\n\n \t\t// Return the exports of the module\n \t\treturn module.exports;\n \t}\n\n\n \t// expose the modules object (__webpack_modules__)\n \t__webpack_require__.m = modules;\n\n \t// expose the module cache\n \t__webpack_require__.c = installedModules;\n\n \t// define getter function for harmony exports\n \t__webpack_require__.d = function(exports, name, getter) {\n \t\tif(!__webpack_require__.o(exports, name)) {\n \t\t\tObject.defineProperty(exports, name, { enumerable: true, get: getter });\n \t\t}\n \t};\n\n \t// define __esModule on exports\n \t__webpack_require__.r = function(exports) {\n \t\tif(typeof Symbol !== 'undefined' && Symbol.toStringTag) {\n \t\t\tObject.defineProperty(exports, Symbol.toStringTag, { value: 'Module' });\n \t\t}\n \t\tObject.defineProperty(exports, '__esModule', { value: true });\n \t};\n\n \t// create a fake namespace object\n \t// mode & 1: value is a module id, require it\n \t// mode & 2: merge all properties of value into the ns\n \t// mode & 4: return value when already ns object\n \t// mode & 8|1: behave like require\n \t__webpack_require__.t = function(value, mode) {\n \t\tif(mode & 1) value = __webpack_require__(value);\n \t\tif(mode & 8) return value;\n \t\tif((mode & 4) && typeof value === 'object' && value && value.__esModule) return value;\n \t\tvar ns = Object.create(null);\n \t\t__webpack_require__.r(ns);\n \t\tObject.defineProperty(ns, 'default', { enumerable: true, value: value });\n \t\tif(mode & 2 && typeof value != 'string') for(var key in value) __webpack_require__.d(ns, key, function(key) { return value[key]; }.bind(null, key));\n \t\treturn ns;\n \t};\n\n \t// getDefaultExport function for compatibility with non-harmony modules\n \t__webpack_require__.n = function(module) {\n \t\tvar getter = module && module.__esModule ?\n \t\t\tfunction getDefault() { return module['default']; } :\n \t\t\tfunction getModuleExports() { return module; };\n \t\t__webpack_require__.d(getter, 'a', getter);\n \t\treturn getter;\n \t};\n\n \t// Object.prototype.hasOwnProperty.call\n \t__webpack_require__.o = function(object, property) { return Object.prototype.hasOwnProperty.call(object, property); };\n\n \t// __webpack_public_path__\n \t__webpack_require__.p = \"/\";\n\n\n \t// Load entry module and return exports\n \treturn __webpack_require__(__webpack_require__.s = \"./styles/dark.css\");\n","// extracted by mini-css-extract-plugin"],"sourceRoot":""}
-35
View File
@@ -1,35 +0,0 @@
'use strict';
System.register(['lodash'], function (_export, _context) {
"use strict";
var _, DatasourceConfigCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_lodash) {
_ = _lodash.default;
}],
execute: function () {
_export('DatasourceConfigCtrl', DatasourceConfigCtrl = function DatasourceConfigCtrl() {
_classCallCheck(this, DatasourceConfigCtrl);
this.current.jsonData = this.current.jsonData || {};
this.current.access = 'direct';
this.current.basicAuth = false;
this.current.withCredentials = true;
this.current.suggestHost = 'localhost';
});
_export('DatasourceConfigCtrl', DatasourceConfigCtrl);
DatasourceConfigCtrl.templateUrl = 'datasource/partials/config.html';
}
};
});
//# sourceMappingURL=config.js.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["../../src/datasource/config.js"],"names":["_","DatasourceConfigCtrl","current","jsonData","access","basicAuth","withCredentials","suggestHost","templateUrl"],"mappings":";;;;;;;;;;;;;;;AAAOA,O;;;sCAGMC,oB,GAEX,gCAAc;AAAA;;AACZ,aAAKC,OAAL,CAAaC,QAAb,GAAwB,KAAKD,OAAL,CAAaC,QAAb,IAAyB,EAAjD;AACA,aAAKD,OAAL,CAAaE,MAAb,GAAsB,QAAtB;AACA,aAAKF,OAAL,CAAaG,SAAb,GAAyB,KAAzB;AACA,aAAKH,OAAL,CAAaI,eAAb,GAA+B,IAA/B;AACA,aAAKJ,OAAL,CAAaK,WAAb,GAA2B,WAA3B;AACD,O;;;;AAEHN,2BAAqBO,WAArB,GAAmC,iCAAnC","file":"config.js","sourcesContent":["import _ from 'lodash';\n\n\nexport class DatasourceConfigCtrl {\n\n constructor() {\n this.current.jsonData = this.current.jsonData || {};\n this.current.access = 'direct';\n this.current.basicAuth = false;\n this.current.withCredentials = true;\n this.current.suggestHost = 'localhost';\n }\n}\nDatasourceConfigCtrl.templateUrl = 'datasource/partials/config.html';\n"]}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -1,710 +0,0 @@
{
"__inputs": [
{
"name": "DS_TIMELY",
"label": "Timely",
"description": "",
"type": "datasource",
"pluginId": "timely-datasource",
"pluginName": "Timely"
}
],
"__requires": [
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "4.4.1"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "panel",
"id": "singlestat",
"name": "Singlestat",
"version": ""
},
{
"type": "datasource",
"id": "timely-datasource",
"name": "Timely",
"version": "1.0.0"
}
],
"annotations": {
"list": []
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"hideControls": false,
"id": null,
"links": [],
"refresh": "1m",
"rows": [
{
"collapse": false,
"height": "150",
"panels": [
{
"cacheTimeout": null,
"colorBackground": false,
"colorValue": false,
"colors": [
"rgba(245, 54, 54, 0.9)",
"rgba(237, 129, 40, 0.89)",
"rgba(50, 172, 45, 0.97)"
],
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"format": "none",
"gauge": {
"maxValue": 100000,
"minValue": 0,
"show": true,
"thresholdLabels": false,
"thresholdMarkers": false
},
"height": "150",
"hideTimeOverride": true,
"id": 8,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 4,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"aggregator": "sum",
"downsampleAggregator": "sum",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "timely.metrics.received",
"refId": "A"
}
],
"thresholds": "",
"timeFrom": null,
"timeShift": "1m",
"title": "Metrics Received Last Minute",
"transparent": true,
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": false,
"colorValue": false,
"colors": [
"rgba(245, 54, 54, 0.9)",
"rgba(237, 129, 40, 0.89)",
"rgba(50, 172, 45, 0.97)"
],
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"format": "none",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "150",
"hideTimeOverride": true,
"id": 9,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 4,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"aggregator": "count",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "count",
"downsampleFillPolicy": "none",
"downsampleInterval": "1s",
"metric": "sys.collectd.cache_size",
"refId": "A",
"tags": {}
}
],
"thresholds": "",
"timeShift": "1m",
"title": "CollectD Instances Reporting",
"transparent": true,
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": false,
"colorValue": false,
"colors": [
"rgba(245, 54, 54, 0.9)",
"rgba(237, 129, 40, 0.89)",
"rgba(50, 172, 45, 0.97)"
],
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"format": "none",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "150",
"hideTimeOverride": true,
"id": 10,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 4,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"aggregator": "sum",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "sum",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "sys.collectd.derive.dropped",
"refId": "A",
"tags": {}
}
],
"thresholds": "",
"timeShift": "1m",
"title": "Metrics Dropped By CollectD",
"transparent": true,
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "New row",
"titleSize": "h6"
},
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"fill": 0,
"grid": {},
"id": 3,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "sum",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "sum",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "timely.keys.metric.inserted",
"refId": "A",
"tags": {
"host": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Timely Metrics Inserted",
"tooltip": {
"msResolution": true,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "none",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"fill": 0,
"grid": {},
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "sum",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "avg",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "timely.keys.meta.inserted",
"refId": "A",
"tags": {
"host": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Timely Meta Inserted",
"tooltip": {
"msResolution": true,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "none",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"fill": 0,
"grid": {},
"id": 5,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "sum",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "sum",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "timely.metrics.received",
"refId": "A",
"tags": {
"host": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Timely Metrics Received",
"tooltip": {
"msResolution": true,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "none",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY}",
"editable": true,
"error": false,
"fill": 0,
"grid": {},
"id": 7,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "none",
"alias": "$tag_host",
"currentTagKey": "",
"currentTagValue": "",
"downsampleAggregator": "sum",
"downsampleFillPolicy": "none",
"downsampleInterval": "1m",
"metric": "sys.collectd.cache_size",
"refId": "A",
"tags": {
"host": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "CollectD Cache Sizes",
"tooltip": {
"msResolution": true,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "New row",
"titleSize": "h6"
}
],
"schemaVersion": 14,
"style": "dark",
"tags": [],
"templating": {
"list": []
},
"time": {
"from": "now-1h",
"to": "now"
},
"timepicker": {
"now": true,
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "browser",
"title": "Timely Status",
"version": 4
}
@@ -1,479 +0,0 @@
{
"__inputs": [
{
"name": "DS_TIMELY-STANDALONE",
"label": "Timely-Standalone",
"description": "",
"type": "datasource",
"pluginId": "timely-datasource",
"pluginName": "Timely"
}
],
"__requires": [
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "4.4.1"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "datasource",
"id": "timely-datasource",
"name": "Timely",
"version": "1.0.0"
}
],
"annotations": {
"list": []
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"hideControls": false,
"id": null,
"links": [],
"refresh": "5s",
"rows": [
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY-STANDALONE}",
"editable": true,
"error": false,
"fill": 1,
"grid": {},
"id": 1,
"legend": {
"alignAsTable": true,
"avg": false,
"current": true,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": true,
"values": true
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "sum",
"disableDownsampling": false,
"downsampleAggregator": "avg",
"downsampleFillPolicy": "none",
"downsampleInterval": "",
"errors": {},
"isCounter": false,
"metric": "timely.metrics.received",
"refId": "A",
"shouldComputeRate": false
},
{
"aggregator": "sum",
"disableDownsampling": false,
"downsampleAggregator": "avg",
"downsampleFillPolicy": "none",
"downsampleInterval": "",
"errors": {},
"metric": "timely.keys.meta.inserted",
"refId": "B"
},
{
"aggregator": "sum",
"disableDownsampling": false,
"downsampleAggregator": "avg",
"downsampleFillPolicy": "none",
"downsampleInterval": "",
"errors": {},
"metric": "timely.keys.metric.inserted",
"refId": "C"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Timely Status",
"tooltip": {
"msResolution": true,
"shared": false,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": "Keys",
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Row",
"titleSize": "h6"
},
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY-STANDALONE}",
"editable": true,
"error": false,
"fill": 1,
"grid": {},
"id": 2,
"legend": {
"alignAsTable": true,
"avg": false,
"current": false,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "none",
"alias": "$tag_host",
"currentTagKey": "",
"currentTagValue": "",
"disableDownsampling": false,
"downsampleAggregator": "max",
"downsampleFillPolicy": "none",
"downsampleInterval": "1s",
"metric": "sys.cpu.user",
"refId": "A",
"tags": {
"host": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "CPU User by Host",
"tooltip": {
"msResolution": false,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY-STANDALONE}",
"editable": true,
"error": false,
"fill": 1,
"grid": {},
"id": 3,
"legend": {
"alignAsTable": true,
"avg": false,
"current": false,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "none",
"alias": "$tag_rack",
"currentTagKey": "",
"currentTagValue": "",
"disableDownsampling": false,
"downsampleAggregator": "max",
"downsampleFillPolicy": "none",
"downsampleInterval": "1s",
"metric": "sys.cpu.user",
"refId": "A",
"tags": {
"rack": ".*"
}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "CPU User by Rack",
"tooltip": {
"msResolution": false,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_TIMELY-STANDALONE}",
"editable": true,
"error": false,
"fill": 1,
"grid": {},
"id": 4,
"legend": {
"alignAsTable": true,
"avg": false,
"current": false,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"aggregator": "none",
"alias": "",
"currentTagKey": "",
"currentTagValue": "",
"disableDownsampling": false,
"downsampleAggregator": "max",
"downsampleFillPolicy": "none",
"downsampleInterval": "1s",
"metric": "sys.cpu.user",
"refId": "A",
"tags": {}
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Cumulative CPU",
"tooltip": {
"msResolution": false,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "New row",
"titleSize": "h6"
}
],
"schemaVersion": 14,
"style": "dark",
"tags": [
"imported"
],
"templating": {
"list": []
},
"time": {
"from": "now-15m",
"to": "now"
},
"timepicker": {
"now": true,
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "browser",
"title": "Timely Standalone Demo",
"version": 6
}
Loaded 100 of 353 files, more files were not shown because too many files have changed in this diff. Show more