1. 技术文章
  2. skywalking系列(2)-在IntelliJ IDEA中通过skywalking源码构建项目

skywalking系列(2)-在IntelliJ IDEA中通过skywalking源码构建项目

由 泄露秘密 提交于 2020-04-25 05:37:07

目的 为了学习skywalking源码的设计，以及更好的了解内部原理，对系统进行扩展，优化。

参考 github上skywalking官方教程:https://github.com/apache/skywalking/blob/master/docs/en/guides/How-to-build.md SkyWalking 6.x 源码分析 —— 调试环境搭建【芋道源码】:http://www.iocoder.cn/SkyWalking/6/build-debugging-environment/

环境准备 操作系统: macOS 10.14.1 skywalking源码版本: github上6.6.0之后的master分支(2020-1-29) IDE: intelliJ IDEA jdk: 1.8 maven: maven 3

构建项目

####1.clone代码 git clone https://github.com/apache/skywalking.git

遇到的问题 (1) 问题：clone代码失败： 解决办法：配置host，可加快网速

219.76.4.4 github-cloud.s3.amazonaws.com
192.30.253.112 github.com
151.101.185.194 github.global.ssl.fastly.net

####2.选择maven方式导入skywalking源码到IDEA

####3.执行git命令 git submodule init

####4.执行git命令 git submodule update 这两步的目的是获取skywalking子模块的源码，子模块包括apm-network、query-graphql-plugin、skywalking-ui、e2e-ttl-es，具体的依赖信息可查看/.gitmodules文件

[submodule "apm-protocol/apm-network/src/main/proto"]
	path = apm-protocol/apm-network/src/main/proto
	url = https://github.com/apache/skywalking-data-collect-protocol.git
[submodule "oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol"]
	path = oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol
	url = https://github.com/apache/skywalking-query-protocol.git
[submodule "skywalking-ui"]
	path = skywalking-ui
	url = https://github.com/apache/skywalking-rocketbot-ui.git
[submodule "test/e2e/e2e-ttl/e2e-ttl-es/src/main/proto"]
	path = test/e2e/e2e-ttl/e2e-ttl-es/src/main/proto
	url = https://github.com/apache/skywalking-data-collect-protocol.git
  • 【敲黑板】该步骤非常重要，不完整执行成功，后续的编译会失败。
  • git submodule update执行很慢，还可能中途中断，但是执行的过程中不要手动中断命令的执行，等10分钟看结果，不然文件可能只下载了一部分，导致莫名其妙的问题。

遇到的问题 (1)git submodule update失败

terrydeMacBook-Pro:skywalking-terry terry$ git submodule init
terrydeMacBook-Pro:skywalking-terry terry$ git submodule update
Submodule path 'oap-server/server-query-plugin/query-graphql-plugin/src/main/resources/query-protocol': checked out '6fc96650acc7f539fcdf6d51648525ce93e5fa1a'
fatal: Needed a single revision
Unable to find current revision in submodule path 'skywalking-ui'

解决办法： 参考：https://www.cnblogs.com/ligun123/p/4139883.html 删除skywalking-ui目录，再执行git submodule update，将会出现如下：

terrydembp:skywalking-github terry$ git submodule update
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 4370 (delta 2), reused 3 (delta 0), pack-reused 4356
Receiving objects: 100% (4370/4370), 13.76 MiB | 36.00 KiB/s, done.
Resolving deltas: 100% (2575/2575), done.
From https://github.com/apache/skywalking-rocketbot-ui
 * [new branch]      develop    -> origin/develop
 * [new branch]      master     -> origin/master
 * [new tag]         v0.9.0     -> v0.9.0
 * [new tag]         v1.0.0     -> v1.0.0
 * [new tag]         v1.0.1     -> v1.0.1
 * [new tag]         v1.0.2     -> v1.0.2
 * [new tag]         v1.0.3     -> v1.0.3
 * [new tag]         v6.1.0     -> v6.1.0
 * [new tag]         v6.2.0     -> v6.2.0
 * [new tag]         v6.3.0     -> v6.3.0
 * [new tag]         v6.4.0     -> v6.4.0
 * [new tag]         v6.5.0     -> v6.5.0
 * [new tag]         v6.6.0     -> v6.6.0
Submodule path 'skywalking-ui': checked out '9ad81a05a2f2969420c1f75769c2b4def42819e4'
Submodule path 'test/e2e/e2e-ttl/e2e-ttl-es/src/main/proto': checked out '7b244ff7ec350910295eee85633e02d92a6f6b1c'

####5.执行mvn命令 ./mvnw clean package -DskipTests

这一步执行时间长短和网速有关系，编译过两次，一次14min、一次1h

执行情况示例：

terrydembp:skywalking-github terry$ ./mvnw clean package -DskipTests
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 50710  100 50710    0     0  20994      0  0:00:02  0:00:02 --:--:-- 20997
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] apm                                                                [pom]
[INFO] apm-commons                                                        [pom]
省略许多行......
省略许多行......
省略许多行......

遇到的问题 (1) ./mvnw clean package -DskipTests无法编译 原因：开了vpn，原以为【翻qiang】会快很多，没想到开了VPN，反而下载不了需要的插件。 解决办法：关闭vpm

(2)执行失败，提示缺少部分类，例如缺少类KeyStringValuePair.java

[ERROR] /Users/terry/Gits/agent/skywalking/skywalking-github/apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/command/TraceIgnoreCommand.java:[39,48] 找不到符号
[ERROR]   符号:   变量 KeyStringValuePair
[ERROR]   位置: 类 org.apache.skywalking.apm.network.trace.component.command.TraceIgnoreCommand
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :apm-network

可能是因为git submodule没有更新到submodule，解决办法是：找到上面提到的项目根目录下的/.gitmodules文件，查看submodule对应的path路径，看是否完整更新下来的源码。 image.png 可以删除这几个目录，然后执行git submodule init、git submodule update，重新更新submodule的文件。

执行成功的结果：

[INFO] configuration-consul ............................... SUCCESS [  2.216 s]
[INFO] server-bootstrap ................................... SUCCESS [  4.166 s]
[INFO] server-starter ..................................... SUCCESS [01:27 min]
[INFO] server-starter-es7 ................................. SUCCESS [  8.393 s]
[INFO] apm-webapp ......................................... SUCCESS [07:15 min]
[INFO] apache-skywalking-apm .............................. SUCCESS [ 45.532 s]
[INFO] apache-skywalking-apm-es7 .......................... SUCCESS [ 22.466 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:01 h
[INFO] Finished at: 2020-01-29T13:03:28+08:00

####6.最终在disk目录下生成了文件：

dist/apache-skywalking-apm-bin.tar.gz
dist/apache-skywalking-apm-bin-es7.tar.gz

这些包和官方的release版本的包是一样的，后续可以自己修改源码进行编译、构建、部署。 image.png

####7.生成源码：

grpc-java and java folders in apm-protocol/apm-network/target/generated-sources/protobuf
grpc-java and java folders in oap-server/server-core/target/generated-sources/protobuf
grpc-java and java folders in oap-server/server-receiver-plugin/receiver-proto/target/generated-sources/protobuf
grpc-java and java folders in oap-server/exporter/target/generated-sources/protobuf
grpc-java and java folders in oap-server/server-configuration/grpc-configuration-sync/target/generated-sources/protobuf
antlr4 folder in oap-server/oal-grammar/target/generated-sources

根据官网找到上面的文件目录， 右键--->Mark Directory As --->Generated Rources Root 文件夹会由黄色变成蓝色

启动：

启动OapServer

执行org.apache.skywalking.oap.server.starter.OAPServerStartUp#main方法，启动server。

注意

  • 6.6.0默认是用es7作为storage module，所以需要部署es7； 如果想用其他storage module，则可以修改/oap-server/server-bootstrap/src/main/resources/application.yml目录下的storage节点的配置。然后rebuild一下该文件，重新编译，在重新启动即可。

我采用的的是ES6，修改skywalking的配置如下，启动ES后，再启动OapServer

storage:
  elasticsearch:
    nameSpace: ${SW_NAMESPACE:""}
    clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
    protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
    #trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}
    #trustStorePass: ${SW_SW_STORAGE_ES_SSL_JKS_PASS:""}
    user: ${SW_ES_USER:""}
    password: ${SW_ES_PASSWORD:""}
    indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}
    indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}
    # Those data TTL settings will override the same settings in core module.
    recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day
    otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day
    monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month
    # Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html
    bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:1000} # Execute the bulk every 1000 requests
    flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests
    concurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests
    resultWindowMaxSize: ${SW_STORAGE_ES_QUERY_MAX_WINDOW_SIZE:10000}
    metadataQueryMaxSize: ${SW_STORAGE_ES_QUERY_MAX_SIZE:5000}
    segmentQueryMaxSize: ${SW_STORAGE_ES_QUERY_SEGMENT_SIZE:200}
    profileTaskQueryMaxSize: ${SW_STORAGE_ES_QUERY_PROFILE_TASK_SIZE:200}
    advanced: ${SW_STORAGE_ES_ADVANCED:""}
#  elasticsearch7:
#    nameSpace: ${SW_NAMESPACE:""}
#    clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
#    protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"http"}
#    #trustStorePath: ${SW_SW_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"}
#    #trustStorePass: ${SW_SW_STORAGE_ES_SSL_JKS_PASS:""}
#    user: ${SW_ES_USER:""}
#    password: ${SW_ES_PASSWORD:""}
#    indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}
#    indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}
#    # Those data TTL settings will override the same settings in core module.
#    recordDataTTL: ${SW_STORAGE_ES_RECORD_DATA_TTL:7} # Unit is day
#    otherMetricsDataTTL: ${SW_STORAGE_ES_OTHER_METRIC_DATA_TTL:45} # Unit is day
#    monthMetricsDataTTL: ${SW_STORAGE_ES_MONTH_METRIC_DATA_TTL:18} # Unit is month
#    # Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html
#    bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:1000} # Execute the bulk every 1000 requests
#    flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests
#    concurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests
#    resultWindowMaxSize: ${SW_STORAGE_ES_QUERY_MAX_WINDOW_SIZE:10000}
#    metadataQueryMaxSize: ${SW_STORAGE_ES_QUERY_MAX_SIZE:5000}
#    segmentQueryMaxSize: ${SW_STORAGE_ES_QUERY_SEGMENT_SIZE:200}
#    advanced: ${SW_STORAGE_ES_ADVANCED:""}

查看ES索引 访问http://localhost:9200/_cat/indices 查看ES索引可以检验skywalking初始化数据是否成功 image.png

启动webApp

执行org.apache.skywalking.apm.webapp.ApplicationStartUp方法 验证 访问http://127.0.0.1:8080/ image.png

启动普通java应用，验证链路追踪

配置应用的VM参数如下，再运行应用：

-Dskywalking.agent.service_name=skywalking-test-local -Dskywalking.collector.backend_service=127.0.0.1:11800 -javaagent:/Users/terry/Gits/agent/skywalking-bin/6.6.0/apache-skywalking-apm-bin/agent/skywalking-agent.jar

image.png

验证 访问一个springmvc的请求localhost:8082/a_testSysLog 参考skywalking image.png

标签
git
skywalking
github
maven
java
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!