com.twitter.heron.metricscachemgr.metricscache

Class CacheCore

java.lang.Object

com.twitter.heron.metricscachemgr.metricscache.CacheCore

public class CacheCore
extends java.lang.Object

Cache Engine to store metrics and exceptions in memory and to respond to query, implementing insertion and selection methods: 1. Storage for metrics: timestamp_start -(tree)-> [sparse: bucket_id -(map)-> metric bucket] 2. Storage for exceptions: idxComponentInstance -(hash)-> exception bucket 3. Index for metrics: a. metricName -(map)-> idxMetricName (int: locator) b. component -(map)-> instance -(map)-> idxComponentInstance (int: locator) bucket_id is Long from idxComponentInstance Integer and idxMetricName Integer combined 4. Index for exceptions: component -(map)-> instance -(map)-> idxComponentInstance (int: locator) 5. Query pattern: component-instance (equality), metricName (equality), timestamp (range) Different from tmaster: 1. order bucket by metric timestamp rather than metric message arriving time 2. free buckets for instances that are gone during scaling process 3. lock for multiple threads Same as tmaster: 1. support same protobuf message/request format

Constructor Summary

Constructors

Constructor and Description
CacheCore(long maxIntervalSecs, long intervalSecs, long maxException) constructor: CacheCore needs two intervals to configure metrics time window and one number to limit exception count

Method Summary

Modifier and Type	Method and Description
void	addMetricException(TopologyMaster.PublishMetrics metrics) compatible with heron::tmaster::TMetricsCollector
boolean	componentInstanceExists(java.lang.String componentName, java.lang.String instanceId)
ExceptionResponse	getExceptions(ExceptionRequest request) for internal process use
MetricResponse	getMetrics(MetricRequest request, MetricsFilter metricNameType) for internal process use
boolean	metricExists(java.lang.String name)
void	purge()
void	startPurge(WakeableLooper wakeableLooper) start purge looper task
void	stopPurge() stop metric purge looper
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CacheCore

public CacheCore(long maxIntervalSecs,
                 long intervalSecs,
                 long maxException)

constructor: CacheCore needs two intervals to configure metrics time window and one number to limit exception count

Parameters:

maxIntervalSecs - metric: cache how long time? in seconds

intervalSecs - metric: purge how often? in seconds

maxException - exception: cache how many?

Method Detail

componentInstanceExists

public boolean componentInstanceExists(java.lang.String componentName,
                                       java.lang.String instanceId)

metricExists

public boolean metricExists(java.lang.String name)

addMetricException

public void addMetricException(TopologyMaster.PublishMetrics metrics)

compatible with heron::tmaster::TMetricsCollector

Parameters:

metrics - The metrics to be added

getMetrics

public MetricResponse getMetrics(MetricRequest request,
                                 MetricsFilter metricNameType)

for internal process use

Parameters:

request -

idxMetricName == null: query all metrics idxMetricName == []: query none metric idxMetricName == [a, b, c .. ]: query metric a, b and c, ..

idxComponentInstance == null: query all components idxComponentInstance == []: query none component idxComponentInstance == [c1->null, ..]: query all instances of c1, .. idxComponentInstance == [c1->[], ..]: query none instance of c1, .. idxComponentInstance == [c1->[a, b, c, ..], ..]: query instance a, b, c, .. of c1, ..

assert: startTime <= endTime

metricNameType - map: metric name to type

Returns:

query result

getExceptions

public ExceptionResponse getExceptions(ExceptionRequest request)

for internal process use

Parameters:

request -

idxComponentInstance == null: query all components idxComponentInstance == []: query none component idxComponentInstance == [c1->null, ..]: query all instances of c1, .. idxComponentInstance == [c1->[], ..]: query none instance of c1, .. idxComponentInstance == [c1->[a, b, c, ..], ..]: query instance a, b, c, .. of c1, ..

Returns:

query result

purge

public void purge()

startPurge

public void startPurge(WakeableLooper wakeableLooper)

start purge looper task

Parameters:

wakeableLooper - the looper to run timer

stopPurge

public void stopPurge()

stop metric purge looper

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object