这两天用golang在写一个监控的agent，发现长时间运行后会有内存泄露的情况，着实让人郁闷半天… 要解决golang泄露的问题，要理解goalng gc那是事情，再就是利用pprof监视golang的运行环境。
这Golang GC垃圾回收我就先不多说了，等我自己深入了解了，会专门找个时间聊这事.
在golang中用来做监控分析的库包，一般用都是pprof库包… pprof可以在两个地方引入：
net/http/pprof runtime/pprof其实net/http/pprof中只是使用runtime/pprof包来进行封装了一下，并在http端口上暴露出来。 runtime/pprof可以用来产生dump文件，再使用go tool pprof来分析这运行日志.
使用net/http/pprof可以做到直接看到当前web服务的状态，包括CPU占用情况和内存使用情况等。
这次重点说些pprof web显示的模式，我自己主要是关注heap,profile两个数据源。
关于golang运行环境heap的信息、内存mem等
http://localhost:7777/debug/pprof/heap
关于profile、CPU计算百分比等
http://localhost:7777/debug/pprof/profile
上面简单介绍了pprof的监控接口，但怎么查看这些数据？ 有这么几个查看方式.
交互模式, 可以通过用help查看pprof各种命令。 每次运行后拿到的数据是固定的，不会动态更新该数据。
go tool pprof http://localhost:7777/debug/pprof/heap go tool pprof http://localhost:7777/debug/pprof/profile
如果你不想使用交互模式，当然这每次都是新数据：
go tool pprof --text http://localhost:7777/debug/pprof/heap
网页查看模式:
http://localhost:7777/debug/pprof/
下面是我随便写的一段伪业务代码. 大家可以长时间运行这段代码，会发现内存在缓慢的增长中.
#xiaorui.cc
package main
import (
"flag"
"fmt"
"io/ioutil"
"log"
"net/http"
_ "net/http/pprof"
"sync"
"time"
)
func counter() {
list := []int{1}
c := 1
for i := 0; i < 10000000; i++ {
httpGet()
c = i + 1 + 2 + 3 + 4 - 5
list = append(list, c)
}
fmt.Println(c)
fmt.Println(list[0])
}
func work(wg *sync.WaitGroup) {
for {
counter()
time.Sleep(1 * time.Second)
}
wg.Done()
}
func httpGet() int {
queue := []string{"start..."}
resp, err := http.Get("http://www.163.com")
if err != nil {
// handle error
}
//defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
// handle error
}
queue = append(queue, string(body))
return len(queue)
}
func main() {
flag.Parse()
//这里实现了远程获取pprof数据的接口
go func() {
log.Println(http.ListenAndServe("localhost:7777", nil))
}()
var wg sync.WaitGroup
wg.Add(10)
for i := 0; i < 100; i++ {
go work(&wg)
}
wg.Wait()
time.Sleep(3 * time.Second)
}
下面是pprof统计的heaq信息
#xiaorui.cc
$ go tool pprof --text http://localhost:7777/debug/pprof/heap
Fetching profile from http://localhost:7777/debug/pprof/heap
Saved profile in /Users/ruifengyun/pprof/pprof.localhost:7777.inuse_objects.inuse_space.024.pb.gz
60.81MB of 60.81MB total ( 100%)
Dropped 62 nodes (cum <= 0.30MB)
flat flat% sum% cum cum%
56.23MB 92.46% 92.46% 56.23MB 92.46% bytes.makeSlice
3.08MB 5.07% 97.53% 3.08MB 5.07% compress/flate.NewReader
0.50MB 0.83% 98.36% 0.50MB 0.83% net/http.(*Transport).dialConn
0.50MB 0.82% 99.18% 0.50MB 0.82% net/http.readTransfer
0.50MB 0.82% 100% 0.50MB 0.82% net/http.NewRequest
0 0% 100% 59.31MB 97.53% bytes.(*Buffer).ReadFrom
0 0% 100% 3.08MB 5.07% compress/gzip.(*Reader).readHeader
0 0% 100% 3.08MB 5.07% compress/gzip.NewReader
0 0% 100% 59.31MB 97.53% io/ioutil.ReadAll
0 0% 100% 59.31MB 97.53% io/ioutil.readAll
0 0% 100% 59.81MB 98.35% main.counter
0 0% 100% 59.81MB 98.35% main.httpGet
0 0% 100% 59.81MB 98.35% main.work
0 0% 100% 0.50MB 0.82% net/http.(*Client).Get
0 0% 100% 0.50MB 0.83% net/http.(*Transport).getConn.func4
0 0% 100% 3.08MB 5.07% net/http.(*bodyEOFSignal).Read
0 0% 100% 3.08MB 5.07% net/http.(*gzipReader).Read
0 0% 100% 0.50MB 0.82% net/http.(*persistConn).readLoop
0 0% 100% 0.50MB 0.82% net/http.(*persistConn).readResponse
0 0% 100% 0.50MB 0.82% net/http.Get
0 0% 100% 0.50MB 0.82% net/http.ReadResponse
0 0% 100% 60.81MB 100% runtime.goexit
另外需要说的一点，pprof可以生成一个svg的矢量图，可以通过这svg图确认代码的流程及调用情况. svg是使用graphviz生成的，mac下直接brew install graphviz就能安装，centos下，yum -y install graphviz .
下面是pprof help的使用文档.
#xiaorui.cc
$ go tool pprof http://localhost:6060/debug/pprof/heap
Fetching profile from http://localhost:6060/debug/pprof/heap
Saved profile in /Users/ruifengyun/pprof/pprof.localhost:6060.inuse_objects.inuse_space.032.pb.gz
Entering interactive mode (type "help" for commands)
(pprof) help
Commands:
cmd [n] [--cum] [focus_regex]* [-ignore_regex]*
Produce a text report with the top n entries.
Include samples matching focus_regex, and exclude ignore_regex.
Add --cum to sort using cumulative data.
Available commands:
callgrind Outputs a graph in callgrind format
disasm Output annotated assembly for functions matching regexp or address
dot Outputs a graph in DOT format
eog Visualize graph through eog
evince Visualize graph through evince
gif Outputs a graph image in GIF format
gv Visualize graph through gv
list Output annotated source for functions matching regexp
pdf Outputs a graph in PDF format
peek Output callers/callees of functions matching regexp
png Outputs a graph image in PNG format
proto Outputs the profile in compressed protobuf format
ps Outputs a graph in PS format
raw Outputs a text representation of the raw profile
svg Outputs a graph in SVG format
tags Outputs all tags in the profile
text Outputs top entries in text form
top Outputs top entries in text form
tree Outputs a text rendering of call graph
web Visualize graph through web browser
weblist Output annotated source in HTML for functions matching regexp or address
peek func_regex
Display callers and callees of functions matching func_regex.
dot [n] [focus_regex]* [-ignore_regex]* [>file]
Produce an annotated callgraph with the top n entries.
Include samples matching focus_regex, and exclude ignore_regex.
For other outputs, replace dot with:
- Graphic formats: dot, svg, pdf, ps, gif, png (use > to name output file)
- Graph viewer: gv, web, evince, eog
callgrind [n] [focus_regex]* [-ignore_regex]* [>file]
Produce a file in callgrind-compatible format.
Include samples matching focus_regex, and exclude ignore_regex.
weblist func_regex [-ignore_regex]*
Show annotated source with interspersed assembly in a web browser.
list func_regex [-ignore_regex]*
Print source for routines matching func_regex, and exclude ignore_regex.
disasm func_regex [-ignore_regex]*
Disassemble routines matching func_regex, and exclude ignore_regex.
tags tag_regex [-ignore_regex]*
List tags with key:value matching tag_regex and exclude ignore_regex.
quit/exit/^D
Exit pprof.
option=value
The following options can be set individually:
cum/flat: Sort entries based on cumulative or flat data
call_tree: Build context-sensitive call trees
nodecount: Max number of entries to display
nodefraction: Min frequency ratio of nodes to display
edgefraction: Min frequency ratio of edges to display
focus/ignore: Regexp to include/exclude samples by name/file
tagfocus/tagignore: Regexp or value range to filter samples by tag
eg "1mb", "1mb:2mb", ":64kb"
functions: Level of aggregation for sample data
files:
lines:
addresses:
unit: Measurement unit to use on reports
Sample value selection by index:
sample_index: Index of sample value to display
mean: Average sample value over first value
Sample value selection by name:
alloc_space for heap profiles
alloc_objects
inuse_space
inuse_objects
total_delay for contention profiles
mean_delay
contentions
: Clear focus/ignore/hide/tagfocus/tagignore
根据pprof的统计信息我们可以找到CPU过度计算及内存泄露的大概的点。 现在越来越觉得Golang gc有些让人摸不清头脑. 看来有必要深入学习Golang gc垃圾回收原理.
END… ….