Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

优化内存使用 #22

Merged
merged 3 commits into from
Dec 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 41 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@

## `profiler` 基于 `pprof` 与 `go trace` 持续性能剖析工具

- 根据配置文件收集目标服务的样本, 收集程序会监听配置文件变化实时更新收集目标
- 根据配置文件收集目标服务的样本, 收集程序会监听配置文件的变化,即时应用变化后的配置文件
- 支持的样本 `trace` `fgprof` `profile` `mutex` `heap` `goroutine` `allocs` `block` `threadcreate`
- 提供图表观测服务性能指标的趋势,找出性能问题的时间点
- 点击图标中的气泡跳转到 `pprof` 与 `trace` 的详细页面进行进一步详细的分析
- 提供 `chart` 观测服务性能指标的趋势,找出性能问题的时间点
- 点击 `chart` 中的气泡跳转到 `pprof` 与 `trace` 的详细页面进行进一步详细的分析

<table>
<tr>
<td width="50%" align="center"><b>pprof图表</b></td>
<td width="50%" align="center"><b>pprof chart</b></td>
<td width="50%" align="center"><b>点击气泡跳转pprof详情</b></td>
</tr>
<tr>
<td><img src="https://xtruth.oss-cn-shenzhen.aliyuncs.com/profiler.png"/></td>
<td><img src="https://xtruth.oss-cn-shenzhen.aliyuncs.com/profiler-pprof.png"/></td>
</tr>
<tr>
<td width="50%" align="center"><b>trace图表</b></td>
<td width="50%" align="center"><b>trace chart</b></td>
<td width="50%" align="center"><b>点击气泡跳转trace详情</b></td>
</tr>
<tr>
Expand Down Expand Up @@ -57,25 +57,27 @@ docker run -d -p 80:80 -v ~/profiler/data/:/profiler/data/ -v ~/profiler/config/

## 抓取配置

需要被收集分析的 `golang` 程序,需要提供 `net/http/pprof` 端点,并配置在 `./collector.yaml` 配置文件中
需要被收集分析的 `golang` 程序,需要提供 `net/http/pprof` 端点,并配置在 `./collector.yaml` 配置文件中。

配置文件可以在线更新,收集程序会监听配置文件的变化,即时应用变化后的配置文件。

### `collector.yaml`

```yaml
collector:
targetConfigs:

profiler-server: # 服务名称
profiler-server: # 目标名称
interval: 15s # 抓取间隔
expiration: 0 # 无过期时间
host: localhost:9000 # 目标服务host
profileConfigs: # 默认抓取 (trace, profile, fgprof, mutex, heap, goroutine, allocs, block, threadcreate)
profileConfigs: # 使用默认配置

server2:
interval: 10s
expiration: 168h # 过期时间7天
host: localhost:9000
profileConfigs: # 覆盖默认配置(trace,fgprof,profile,heap)的部分字段
profileConfigs: # 覆盖部分默认配置字段
trace:
enable: false
fgprof:
Expand All @@ -89,45 +91,34 @@ collector:
```

### `profileConfigs` 默认配置
```go
func defaultProfileConfigs() map[string]ProfileConfig {
return map[string]ProfileConfig{
"profile": {
Path: "/debug/pprof/profile?seconds=10",
Enable: utils.BoolPtr(true),
},
"fgprof": {
Path: "/debug/fgprof?seconds=10",
Enable: utils.BoolPtr(true),
},
"trace": {
Path: "/debug/pprof/trace?seconds=10",
Enable: utils.BoolPtr(true),
},
"mutex": {
Path: "/debug/pprof/mutex",
Enable: utils.BoolPtr(true),
},
"heap": {
Path: "/debug/pprof/heap",
Enable: utils.BoolPtr(true),
},
"goroutine": {
Path: "/debug/pprof/goroutine",
Enable: utils.BoolPtr(true),
},
"allocs": {
Path: "/debug/pprof/allocs",
Enable: utils.BoolPtr(true),
},
"block": {
Path: "/debug/pprof/block",
Enable: utils.BoolPtr(true),
},
"threadcreate": {
Path: "/debug/pprof/threadcreate",
Enable: utils.BoolPtr(true),
},
}
}

```yaml
profileConfigs:
profile:
Path: "/debug/pprof/profile?seconds=10"
Enable: true
fgprof:
Path: "/debug/fgprof?seconds=10"
Enable: true
trace:
Path: "/debug/pprof/trace?seconds=10"
Enable: true
mutex:
Path: "/debug/pprof/mutex"
Enable: true
heap:
Path: "/debug/pprof/heap"
Enable: true
goroutine:
Path: "/debug/pprof/goroutine"
Enable: true
allocs:
Path: "/debug/pprof/allocs"
Enable: true
block:
Path: "/debug/pprof/block"
Enable: true
threadcreate:
Path: "/debug/pprof/threadcreate"
Enable: true
```
10 changes: 5 additions & 5 deletions pkg/collector/collector.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ import (

"github.com/google/pprof/profile"
"github.com/sirupsen/logrus"
"github.com/xyctruth/profiler/pkg/internal/v1175/trace"
"github.com/xyctruth/profiler/pkg/storage"
)

Expand Down Expand Up @@ -213,10 +212,11 @@ func (collector *Collector) analysis(profileType string, profileBytes []byte) er
func (collector *Collector) analysisTrace(profileType string, profileBytes []byte) error {
buf := &bytes.Buffer{}
buf.Write(profileBytes)
_, err := trace.Parse(buf, collector.TargetName)
if err != nil {
return err
}

//_, err := trace.Parse(buf, collector.TargetName)
//if err != nil {
// return err
//}

profileID, err := collector.store.SaveProfile(profileBytes, collector.Expiration)
if err != nil {
Expand Down
6 changes: 5 additions & 1 deletion pkg/storage/badger/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ type store struct {
}

func NewStore(path string) storage.Store {
db, err := badger.Open(badger.DefaultOptions(path).WithLoggingLevel(3).WithBypassLockGuard(true))
db, err := badger.Open(
badger.DefaultOptions(path).
WithLoggingLevel(3).
WithBypassLockGuard(true))

if err != nil {
panic(err)
}
Expand Down