This issue recommends Plumelog, an easy-to-use Java distributed logging component.

system introduction

1. Distributed log system without code intrusion, based on log4j, log4j2, logback to collect logs, set the link ID, easy to query associated logs
   Based on elasticsearch as query engine
   High throughput, high query efficiency
   The whole process does not occupy the application local disk space, maintenance-free; The project is transparent and does not affect the operation of the project
   No need to modify the old project, introduce direct use, support dubbo, support springcloud

system architecture

• plumelog-core contains a log collector that collects logs and pushes them to kafka, redis, and other queues
  plumelog-server is responsible for asynchronously writing logs from the queue to elasticsearch
  plumelog-demo Springboot-based use case
  Below is the full feature map, the red part is 4.0 content, currently under development

use

First, the server installation

Step 1: Install redis or kafka (redis is sufficient for most companies)

Step 2: Install elasticsearch

Step 3: download the installation package, plumelog – server download address: https://gitee.com/plumeorg/plumelog/releases

Note: UI and server are merged after 3.1, and the plumelog-ui project can be deployed without deployment

Step 4: Configure plumelog-server and start it

Step 5: Background query

More information:
https://gitee.com/plumeorg/plumelog/blob/master/HELP.md

Server configuration file
Plumelog – server/application. The properties, rounding

        spring.application.name=plumelog_server        server.port=8891        spring.thymeleaf.mode=LEGACYHTML5        spring.mvc.view.prefix=classpath:/templates/        spring.mvc.view.suffix=.html        spring.mvc.static-path-pattern=/plumelog/**        #value6172.16.247.143:9092,172.16.2479092#plumelog.kafka.kafka#If using redis with a password, enable the following configuration        #plumelog.queue.redis.redisPassWord=plumelog        #plumelog.queue.redis.red#Sentinel mode requires filling in masterName        #plumelog.queue.redis.sentinel.masterName=plumelog                #Redis standalone mode and kafka mode must configure the management redis address. Redis cluster mode does not need to configure the management redis address, and configuring it will have no effect        plumelog.redis.redi#If using redis with a password, enable the following configuration        #plumelog.redis.redisPassWord=plumelog                #If using rest, enable the following configuration        #plumelog.rest.restUrl=http://below configuration        #plumelog.queue.redis.redisPassWord=plumelog        #plumelog.queue.redis.redisDb=0#Sentinel mode requires filling in masterName        #plumelog.queue.redis.sentinel.maste#Elasticsearch related configuration, separate multiple hosts with commas        plumelog.es.esHosts=configure the management redis address, configuring it will have no effect        plumelog.redis.redisHost=isHost=127.0.0<#ES7.* has removed the indexjs-number">.1:127.0.0o verify hostname        #plumelog.es.hostnameVerification=lass="hljs-number">#Pull interval, not effective for kafka        plumelog.interval=n>field, so if using es7, no need to configure this,an>        #Pull interval, not effective for kafka        plumelog.interval=n>字段，所以如果是es7不用配置这个，<#plumelog-ui address. If not configured, links cannot be clicked in alarm messages        plumelog.ui.url=https://log        plumelog.es.shards=5        plumelog.es.replicas=30<#Log retention days, confiGIt is recommended to start hourly mode     #Link retention days, configuremodel=day        #Specify the timezone for index creation or leave unconfigured for permanent retention by default        admin.ype.zoneId=GMT+nfiguremodel=day        #指定索引建立的时区 or leave unconfigured for permanent retention by default        admin.ype.zoneId=GMT+100        #拉取时间间隔，kafka不生效        plume#Management password, required when manually deleting logs        admin.password=    #plumelog-ui的地址#Log retention days, configure        plumelog.ui.uror leave unconfigured for permanent retention by default        admin.js-number">127.0.18891                #管理密码，手动删除日志的#Login username and password, empty means no login interface        login.username=admin        login.password=admin0Or do not configure, and it will be retained permanently by default.log.keepDays=30               #链路保留天数,配置0Or not configured, retained permanently by default.log.trace.keepDays=30                #登录用户名密码，为空没有登录界面        login.username=admin        login.password=admin

Second, client use

The client is used in the project, the non-Maven project downloads the dependency package (
https://gitee.com/frankchenlong/plumelog/releases) under your own lib used directly, package can be used to remove the repeat, then configure log4j can collect log.

logback is recommended, especially springboot and springcloud projects. Note: There are bugs in logback 3.2, please use 3.2.1 to fix it or later.

Introducing Maven dependencies

   <>   </dependency>

Configure the log4j configuration file to add the following Appender as an#kafka as middleware log4j.appender.L=com.plumelog.log4j.appender.KafkaAppender #appName system name (define it yourself) log4j.appender.L.appName=plumelog log4j.appender.L.env=${spring.profiles.active} log4j.appender.L.kafkaHosts=172.16.247.143:9092,172.16.247.60:9092,172.16.247.64:9092 #redis as middleware log4j.appender.L=com.plumelog.log4j.appender.RedisAppender log4j.appender.L.appName=plumelog log4j.appender.L.env=${spring.profiles.active} log4j.appender.L.redisHost=172.16.249.72:6379 #if redis has no password, leave this empty or omit it #log4j.appender.L.redisAuth=123456.appName=plumelog log4j.appender.L.env=${spring.profiles.active} log4j.appender.L.kafkaHosts=172.16.247.143:9092,172.16.247.60:9092,172.16.247.64:9092 #redis做为中间件 log4j.appender.L=com.plumelog.log4j.appender.RedisAppender log4j.appender.L.appName=plumelog log4j.appender.L.env=${spring.profiles.active} log4j.appender.L.redisHost=172.16.249.72:6379 #redis没有密码这一项为空或者不需要 #log4j.appender.L.redisAuth=123456

Describes the client configuration

traceID generation configuration

Non-springboot,cloud projects that //Set the TraceID value; without this point, the link ID will not exist return true; } } //Annotation-configured filter example @Bean public FilterRegistrationBean filterRegistrationBean1() { FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean(); filterRegistrationBean.setFilter(initCustomFilter()); filterRegistrationBean.addUrlPatterns(“/*”); filterRegistrationBean.setOrder(Integer.MIN_VALUE); return filterRegistrationBean; } @Bean public Filter initCustomFilter() { return new TraceIdFilter(); }ride public boolean

spring boot,spring cloud If sleuth is introduced and feign calls are used between projects, traceId can be passed across services by itself

        <dependency>            &"Sub-thread log display" class="hljs-name">groupId>            <artifactId>spring-cloud-starter-sleuth</artifactId>            <version>2.2.7.RELEASE</version>        </dependency>

TraceId is passed across threads

If thread pools are not used, no special processing is required. If thread pools are used”tankSay =》I amSaves the need to decorate each Runnable and Callable when passing them to the thread pool; this logic can be done in the thread pool executorService.execute(ified thread pool

        private static ExecutorService executorService = TtlExecutors.getTtlExecutorService(                    new ThreadPoolExecutor(8, 8,0L, TimeUnit.MILLISECONDS,new LinkedBlockingQueue<Runnable>()));          //Eliminate the need to decorate every Runnable and Callable when passing them into the thread pool; this logic can be handled within the thread pool itself.                executorService.execute(() -> {                      logger.info();          });

Modifies Runnable and Callable

        private static ThreadPoolExecutor threadPoolExecutor= ThreadPoolUtil.getPool(4, 8, 5000);                threadPoolExecutor.execute(TtlRunnable.get(() -> {                   TraceId.logTraceID.get();                   logger.info("tankSay =》我是子线程的日志！{}", TraceId.logTraceID.get());         }));