Java Spring Monitoring & Logging

Main Topics: True old style monitoring with Actuator and Spring Boot Admin
Google cloud logging and monitoring through UI
ELK stack - Elasticsearch, Logstash, Kibana To use Spring Security in web applications, we can get started with the simple annotation @EnableWebSecurity.
Feature Name Description
Actuator
Spring Boot Actuator provides several endpoints, both as HTTP and as JMX MBeans, that let you peek into the inner workings of a Spring Boot application.
Like: /beans, /conditions (autoconfig), /env (incl profiles), /health, /heapdump, /httptrace, /info, /loggers, /mappings, /metrics, /scheduledtasks, /threaddump
Disable / Enable
Base path could be changed with management.endpoints.web.base-path
Most Actuator endpoints are disabled by default, but can be selectively exposed by setting management.endpoints.web.exposure.include and management .endpoints.web.exposure.exclude.
Change Some endpoints, such as the /loggers and /env endpoints, allow for write operations to change a running application’s configuration on the fly
Custom info Details regarding an application’s build and Git commit can be exposed in the /info endpoint.
Can customize by properties, by maven plugin in pom.xml (build info etc.) or in program: Customize
@Component
public class TacoCountInfoContributor implements InfoContributor {
	private TacoRepository tacoRepo; // get info from repository
	public TacoCountInfoContributor(TacoRepository tacoRepo) {
		this.tacoRepo = tacoRepo;
	}
	@Override
	public void contribute(Builder builder) {
		long tacoCount = tacoRepo.count();		// add info to endpoint
		Map tacoMap = new HashMap();
		tacoMap.put("count", tacoCount);
		builder.withDetail("taco-stats", tacoMap);
	}
}
Health
Details: management.endpoint.health.show-details
  • UP—The external system is up and is reachable.
  • DOWN—The external system is down or unreachable (one or more health indicators are DOWN).
  • UNKNOWN—The status of the external system is unclear.
  • OUT_OF_SERVICE—The external system is reachable but is currently unavailable (one or more health indicators are OUT_OF_SERVICE).
Custom health An application’s health can be influenced by a custom health indicator, tracking the health of an externally integrated application Customize
@Component
public class WackoHealthIndicator
 implements HealthIndicator {
 @Override
 public Health health() {
	 int hour = Calendar.getInstance().get(Calendar.HOUR_OF_DAY);
	 if (hour > 12) {
		return Health
				.outOfService()
				.withDetail("reason", "I'm out of service after lunchtime")
				.withDetail("hour", hour)
				.build();
	}
	return Health.up().withDetail("reason", "All is good!").build();
}
Custom metrics Custom application metrics can be registered through Micrometer, which affords Spring Boot applications instant integration with several popular metrics engines such as Datadog, New Relic, and Prometheus
Customize
import io.micrometer.core.instrument.MeterRegistry;
@Component
public class TacoMetrics extends AbstractRepositoryEventListener {
	private MeterRegistry meterRegistry;

	 public TacoMetrics(MeterRegistry meterRegistry) {
		this.meterRegistry = meterRegistry;
	 }
	 @Override
	 protected void onAfterCreate(Taco taco) {
		 List ingredients = taco.getIngredients();
		 for (Ingredient ingredient : ingredients) {
			 meterRegistry.counter("tacocloud",
			 "ingredient", ingredient.getId()).increment();
		 }
	 }
}
Security Actuator web endpoints can be secured using Spring Security, much like any other endpoint in a Spring web application. Configure
@Override
protected void configure(HttpSecurity http) throws Exception {
	 http.authorizeRequests()
	 .antMatchers("/actuator/**").hasRole("ADMIN").and().httpBasic();
	
	// OR 
	 http.requestMatcher(
		EndpointRequest.toAnyEndpoint()  // no need to hardcode basic url
			.excluding("health", "info"))
	 .authorizeRequests()
	 .anyRequest().hasRole("ADMIN").and().httpBasic();	
}
Spring Boot Admin
3rd party tool to monitor services using actuator endpoints
Dependency
 <groupId>de.codecentric</groupId>
 <artifactId>spring-boot-admin-starter-server</artifactId>
Could be registered directly by dependency or through Eureka
Logins could be transferred also
JMX ?

Goggle Cloud tools

Google Cloud kubernetes as well as other Cloud providers let you explore the logs for each pod.
  • See the log for each pod
  • every Request has ID (sleught?) so you can search log for ID and trace request
Monitoring - GKE Dashboard
  • Metrics for each pod
  • CPU, memory etc.
History
  • History of deployments through cubectl commands
Readiness, Liveliness probes
  • can be enabled through Actuator
  • Kubernetes could be configured to use them to reload pod or not send traffic.
  • Configured in deployment yaml file

ELK Stack

ELK Stack - Elasticsearch, Logstash, Kibana
  • Elasticsearch is a NoSQL database that is based on the Lucene search engine.
  • Logstash is a log pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets. It is a dynamic data collection pipeline with an extensible plugin ecosystem and strong Elasticsearch synergy
  • Kibana is a visualization UI layer that works on top of Elasticsearch.
These three projects are used together for log analysis in various environments. So Logstash collects and parses logs, Elastic search indexes and store this information while Kibana provides a UI layer that provide actionable insights. Picture