Merge branch 'master' of https://github.com/xtaci/kcptun

同步主线，增加TCP选项！

Author: flynaj

.gitignore

@@ -24,3 +24,5 @@ _testmain.go
 *.prof
 client/client
 server/server
+build/*
+.DS_Store

.travis.yml

@@ -1,8 +1,8 @@
 language: go
 go:
-    - 1.9.x
     - 1.10.x
     - 1.11.x
+    - 1.12.x
 before_install:
     - go get github.com/mattn/goveralls
     - go get golang.org/x/tools/cmd/cover

Dockerfile

@@ -2,10 +2,10 @@ FROM golang:alpine as builder
 MAINTAINER xtaci <daniel820313@gmail.com>
 RUN apk update && \
     apk upgrade && \
-    apk add git
+    apk add git gcc libc-dev linux-headers
 RUN go get -ldflags "-X main.VERSION=$(date -u +%Y%m%d) -s -w" github.com/xtaci/kcptun/client && go get -ldflags "-X main.VERSION=$(date -u +%Y%m%d) -s -w" github.com/xtaci/kcptun/server
 
-FROM alpine:3.6
+FROM alpine:3.9
 COPY --from=builder /go/bin /bin
 EXPOSE 29900/udp
 EXPOSE 12948

README.md

@@ -69,7 +69,15 @@ All precompiled releases are genereated from `build-release.sh` script.
 
 ### Performance
 
-<img src="fast.png" alt="fast.com" height="256px" />       
+<img src="fast.png" alt="fast.com" height="256px" />  
+
+![bandwidth](bw.png)
+
+![flame](flame.png)
+
+> Practical bandwidth graph with parameters:  -mode fast3 -ds 10 -ps 3
+
+
 
 ### Basic Tuning Guide
 
@@ -92,7 +100,15 @@ All precompiled releases are genereated from `build-release.sh` script.
 
 > *fast3 > fast2 > fast > normal > default*
 
+#### HOLB
+
+Since streams are multiplexed into a single physical channel, head of line blocking may appear under certain circumstances, by
+increasing `-smuxbuf` to a larger value (default 4MB) may mitigate this problem, obviously this will costs more memory.
 
+#### Slow Devices
+
+kcptun made use of **ReedSolomon-Codes** to recover lost packets, which requires massive amount of computation, a low-end ARM device cannot satisfy kcptun well. To unleash the full potential of kcptun, a multi-core x86 homeserver CPU like AMD Opteron is recommended.
+If you insist on running under some ARM routers, you'd better turn off `FEC` and use `salsa20` as the encryption method.
 
 ### Expert Tuning Guide
 
@@ -103,15 +119,15 @@ All precompiled releases are genereated from `build-release.sh` script.
 #### Usage
 
 ```
-$ ./client_darwin_amd64 -h
+xtaci@gw:~$ ./client_linux_amd64 -h
 NAME:
    kcptun - client(with SMUX)
 
 USAGE:
-   client_darwin_amd64 [global options] command [command options] [arguments...]
+   client_linux_amd64 [global options] command [command options] [arguments...]
 
 VERSION:
-   20180922
+   20190409
 
 COMMANDS:
      help, h  Shows a list of commands or help for one command
@@ -132,25 +148,26 @@ GLOBAL OPTIONS:
    --parityshard value, --ps value  set reed-solomon erasure coding - parityshard (default: 3)
    --dscp value                     set DSCP(6bit) (default: 0)
    --nocomp                         disable compression
-   --sockbuf value                  (default: 4194304)
-   --keepalive value                (default: 10)
+   --sockbuf value                  per-socket buffer in bytes (default: 4194304)
+   --smuxbuf value                  the overall de-mux buffer in bytes (default: 4194304)
+   --keepalive value                seconds between heartbeats (default: 10)
    --snmplog value                  collect snmp to file, aware of timeformat in golang, like: ./snmp-20060102.log
    --snmpperiod value               snmp collect period, in seconds (default: 60)
    --log value                      specify a log file to output, default goes to stderr
    --quiet                          to suppress the 'stream open/close' messages
    -c value                         config from json file, which will override the command from shell
    --help, -h                       show help
    --version, -v                    print the version
-
-$ ./server_darwin_amd64 -h
+   
+xtaci@gw:~$ ./server_linux_amd64 -h
 NAME:
    kcptun - server(with SMUX)
 
 USAGE:
-   server_darwin_amd64 [global options] command [command options] [arguments...]
+   server_linux_amd64 [global options] command [command options] [arguments...]
 
 VERSION:
-   20180922
+   20190409
 
 COMMANDS:
      help, h  Shows a list of commands or help for one command
@@ -168,8 +185,9 @@ GLOBAL OPTIONS:
    --parityshard value, --ps value  set reed-solomon erasure coding - parityshard (default: 3)
    --dscp value                     set DSCP(6bit) (default: 0)
    --nocomp                         disable compression
-   --sockbuf value                  (default: 4194304)
-   --keepalive value                (default: 10)
+   --sockbuf value                  per-socket buffer in bytes (default: 4194304)
+   --smuxbuf value                  the overall de-mux buffer in bytes (default: 4194304)
+   --keepalive value                seconds between heartbeats (default: 10)
    --snmplog value                  collect snmp to file, aware of timeformat in golang, like: ./snmp-20060102.log
    --snmpperiod value               snmp collect period, in seconds (default: 60)
    --pprof                          start profiling server on :6060
@@ -249,11 +267,17 @@ aes-128-cfb     847216.79k   850770.86k   853712.05k   859912.39k   854565.80k
 The encrytion performance in kcptun is as fast as in openssl library(if not faster).
 
 
-#### Memory Usage Control
+#### Memory Control
 
 Routers, mobile devices are susceptible to memory consumption; by setting GOGC environment(eg: GOGC=20) will make the garbage collector to recycle faster.
 Reference: https://blog.golang.org/go15gc
 
+Primary memory allocation are done from a global buffer pool *xmit.Buf*, in kcp-go, when we need to allocate some bytes, we can get from that pool, and a *fixed-capacity* 1500 bytes(mtuLimit) will be returned, the *rx queue*, *tx queue* and *fec queue* all receive bytes from there, and they will return the bytes to the pool after using to prevent *unnecessary zer0ing* of bytes. 
+The pool mechanism maintained a *high watermark* for slice objects, these *in-flight* objects from the pool will survive from the perodical garbage collection, meanwhile the pool kept the ability to return the memory to runtime if in idle, `-sndwnd`,`-rcvwnd`,`-ds`, `-ps`, these parameters affect this *high watermark*, the larger the value, the bigger the memory consumption will be.
+
+`-smuxbuf` also affects the maximum memory consumption, this parameter maintains a subtle balance between *concurrency* and *resource*, you can increase this value(default 4MB) to boost concurrency if you have many clients to serve and you get a powerful server at the same time, and also you can decrease this value to serve only 1 or 2 clients and hope this program can run under some embeded SoC system with limited memory and only you can access. (Notice that the `-smuxbuf` value is not proprotional to concurrency, you need to test.)
+
+
 #### Compression
 
 kcptun has builtin snappy algorithms for compressing streams:
@@ -274,29 +298,30 @@ Compression is enabled by default, you can disable it by setting ```-nocomp``` o
 #### SNMP
 
 ```go
-// Snmp defines network statistics indicator
 type Snmp struct {
-    BytesSent        uint64 // raw bytes sent
-    BytesReceived    uint64
-    MaxConn          uint64
-    ActiveOpens      uint64
-    PassiveOpens     uint64
-    CurrEstab        uint64 // count of connections for now
-    InErrs           uint64 // udp read errors
+    BytesSent        uint64 // bytes sent from upper level
+    BytesReceived    uint64 // bytes received to upper level
+    MaxConn          uint64 // max number of connections ever reached
+    ActiveOpens      uint64 // accumulated active open connections
+    PassiveOpens     uint64 // accumulated passive open connections
+    CurrEstab        uint64 // current number of established connections
+    InErrs           uint64 // UDP read errors reported from net.PacketConn
     InCsumErrors     uint64 // checksum errors from CRC32
-    KCPInErrors      uint64 // packet iput errors from kcp
-    InSegs           uint64
-    OutSegs          uint64
-    InBytes          uint64 // udp bytes received
-    OutBytes         uint64 // udp bytes sent
-    RetransSegs      uint64
-    FastRetransSegs  uint64
-    EarlyRetransSegs uint64
+    KCPInErrors      uint64 // packet iput errors reported from KCP
+    InPkts           uint64 // incoming packets count
+    OutPkts          uint64 // outgoing packets count
+    InSegs           uint64 // incoming KCP segments
+    OutSegs          uint64 // outgoing KCP segments
+    InBytes          uint64 // UDP bytes received
+    OutBytes         uint64 // UDP bytes sent
+    RetransSegs      uint64 // accmulated retransmited segments
+    FastRetransSegs  uint64 // accmulated fast retransmitted segments
+    EarlyRetransSegs uint64 // accmulated early retransmitted segments
     LostSegs         uint64 // number of segs infered as lost
     RepeatSegs       uint64 // number of segs duplicated
     FECRecovered     uint64 // correct packets recovered from FEC
     FECErrs          uint64 // incorrect packets recovered from FEC
-    FECSegs          uint64 // FEC segments received
+    FECParityShards  uint64 // FEC segments received
     FECShortShards   uint64 // number of data shards that's not enough for recovery
 }
 ```
@@ -339,6 +364,7 @@ The parameters below **MUST** be **IDENTICAL** on **BOTH** side:
 1. http://http2.github.io/ -- What is HTTP/2?
 1. http://www.lartc.org/ -- Linux Advanced Routing & Traffic Control
 1. https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem -- Noisy channel coding theorem
+1. https://zhuanlan.zhihu.com/p/53849089 -- kcptun开发小记
 
 ### Donate 
 

build-release-cgo.sh

@@ -0,0 +1,139 @@
+#!/bin/bash
+
+BUILD_DIR=$(dirname "$0")/build
+mkdir -p $BUILD_DIR
+cd $BUILD_DIR
+
+sum="sha1sum"
+
+if [ "$GO111MODULE" != "on" ]; then
+	echo "GO111MODULE is off"
+else
+	echo "GO111MODULE is on"
+fi 
+
+echo "Prerequisites for cross-compiling were written in build-release.sh"
+
+# required library for cross-compiling
+# sudo apt-get install -y automake autogen build-essential ca-certificates   gcc-5-arm-linux-gnueabi g++-5-arm-linux-gnueabi libc6-dev-armel-cross   gcc-5-arm-linux-gnueabihf g++-5-arm-linux-gnueabihf libc6-dev-armhf-cross gcc-5-aarch64-linux-gnu g++-5-aarch64-linux-gnu libc6-dev-arm64-cross  gcc-5-mips-linux-gnu g++-5-mips-linux-gnu libc6-dev-mips-cross gcc-5-mipsel-linux-gnu g++-5-mipsel-linux-gnu libc6-dev-mipsel-cross  gcc-5-mips64-linux-gnuabi64 g++-5-mips64-linux-gnuabi64 libc6-dev-mips64-cross  gcc-5-mips64el-linux-gnuabi64 g++-5-mips64el-linux-gnuabi64 libc6-dev-mips64el-cross  gcc-5-multilib g++-5-multilib gcc-mingw-w64 g++-mingw-w64 clang llvm-dev   libtool libxml2-dev uuid-dev libssl-dev swig openjdk-8-jdk pkg-config patch  make xz-utils cpio wget zip unzip p7zip git mercurial bzr texinfo help2man --no-install-recommends
+
+# if error message:
+#     /usr/include/linux/errno.h:1:23: fatal error: asm/errno.h: No such file or directory
+# try:
+#    ln -s /usr/include/asm-generic /usr/include/asm
+
+if ! hash sha1sum 2>/dev/null; then
+	if ! hash shasum 2>/dev/null; then
+		echo "I can't see 'sha1sum' or 'shasum'"
+		echo "Please install one of them!"
+		exit
+	fi
+	sum="shasum"
+fi
+
+UPX=false
+if hash upx 2>/dev/null; then
+	UPX=true
+fi
+
+VERSION=`date -u +%Y%m%d`
+LDFLAGS="-X main.VERSION=$VERSION -s -w"
+LDFLAGS_LINUX='-X main.VERSION='$VERSION' -s -w -linkmode "external" -extldflags "-static"'
+LDFLAGS_LINUX32='-X main.VERSION='$VERSION' -s -w -linkmode "external" -extldflags "-static -m32 -L/usr/lib32"'
+echo "-ldflag for linux/amd64:" $LDFLAGS_LINUX
+echo "-ldflag for linux/386:" $LDFLAGS_LINUX32
+echo "-ldflag for other:" $LDFLAGS
+
+echo "=== Building ==="
+
+# 386
+OSES=(linux windows)
+for os in ${OSES[@]}; do
+	suffix=""
+	if [ "$os" == "windows" ]
+	then
+		suffix=".exe"
+	fi
+
+	if [ "$os" == "linux" ];then
+		CC=gcc-5 CGO_ENABLED=1 GOOS=$os GOARCH=386 CGO_CFLAGS="-m32 -L/usr/lib32" CGO_CXXFLAGS="-m32 -L/usr/lib32" go build -ldflags "$LDFLAGS_LINUX32" -o client_${os}_386${suffix} github.com/flynaj/kcptun/client
+		CC=gcc-5 CGO_ENABLED=1 GOOS=$os GOARCH=386 CGO_CFLAGS="-m32 -L/usr/lib32" CGO_CXXFLAGS="-m32 -L/usr/lib32" go build -ldflags "$LDFLAGS_LINUX32" -o server_${os}_386${suffix} github.com/flynaj/kcptun/server
+	else 
+		CGO_ENABLED=0 GOOS=$os GOARCH=386 go build -ldflags "$LDFLAGS" -o client_${os}_386${suffix} github.com/flynaj/kcptun/client
+		CGO_ENABLED=0 GOOS=$os GOARCH=386 go build -ldflags "$LDFLAGS" -o server_${os}_386${suffix} github.com/flynaj/kcptun/server
+	fi
+
+	if $UPX; then upx -9 client_${os}_386${suffix} server_${os}_386${suffix};fi
+	tar -zcf kcptun-${os}-386-$VERSION.tar.gz client_${os}_386${suffix} server_${os}_386${suffix}
+	$sum kcptun-${os}-386-$VERSION.tar.gz
+done
+
+# AMD64 
+OSES=(linux darwin windows freebsd)
+for os in ${OSES[@]}; do
+	suffix=""
+	if [ "$os" == "windows" ]
+	then
+		suffix=".exe"
+	fi
+
+	if [ "$os" == "linux" ];then
+		CC=gcc-5 CGO_ENABLED=1 GOOS=$os GOARCH=amd64 go build -ldflags "$LDFLAGS_LINUX" -o client_${os}_amd64${suffix} github.com/flynaj/kcptun/client
+		CC=gcc-5 CGO_ENABLED=1 GOOS=$os GOARCH=amd64 go build -ldflags "$LDFLAGS_LINUX" -o server_${os}_amd64${suffix} github.com/flynaj/kcptun/server
+	else 
+		CGO_ENABLED=0 GOOS=$os GOARCH=amd64 go build -ldflags "$LDFLAGS" -o client_${os}_amd64${suffix} github.com/flynaj/kcptun/client
+		CGO_ENABLED=0 GOOS=$os GOARCH=amd64 go build -ldflags "$LDFLAGS" -o server_${os}_amd64${suffix} github.com/flynaj/kcptun/server
+	fi
+
+	if $UPX; then upx -9 client_${os}_amd64${suffix} server_${os}_amd64${suffix};fi
+	tar -zcf kcptun-${os}-amd64-$VERSION.tar.gz client_${os}_amd64${suffix} server_${os}_amd64${suffix}
+	$sum kcptun-${os}-amd64-$VERSION.tar.gz
+done
+
+# ARM-5
+#CC=arm-linux-gnueabi-gcc-5 GOOS=linux GOARCH=arm GOARM=5 CGO_ENABLED=1 CGO_CFLAGS="-march=armv5" CGO_CXXFLAGS="-march=armv5" go install std
+CC=arm-linux-gnueabi-gcc-5 CXX=arm-linux-gnueabi-g++-5 GOOS=linux GOARCH=arm GOARM=5 CGO_ENABLED=1 CGO_CFLAGS="-march=armv5" CGO_CXXFLAGS="-march=armv5" go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_arm5  github.com/flynaj/kcptun/client
+CC=arm-linux-gnueabi-gcc-5 CXX=arm-linux-gnueabi-g++-5 GOOS=linux GOARCH=arm GOARM=5 CGO_ENABLED=1 CGO_CFLAGS="-march=armv5" CGO_CXXFLAGS="-march=armv5" go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_arm5  github.com/flynaj/kcptun/server
+if $UPX; then upx -9 client_linux_arm5 server_linux_arm5;fi
+tar -zcf kcptun-linux-arm5-$VERSION.tar.gz client_linux_arm5 server_linux_arm5
+$sum kcptun-linux-arm5-$VERSION.tar.gz
+
+# ARM-6
+#CC=arm-linux-gnueabi-gcc-5 GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CGO_CFLAGS="-march=armv6" CGO_CXXFLAGS="-march=armv6" go install std
+CC=arm-linux-gnueabi-gcc-5 CXX=arm-linux-gnueabi-g++-5 GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CGO_CFLAGS="-march=armv6" CGO_CXXFLAGS="-march=armv6" go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_arm6 github.com/flynaj/kcptun/client
+CC=arm-linux-gnueabi-gcc-5 CXX=arm-linux-gnueabi-g++-5 GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CGO_CFLAGS="-march=armv6" CGO_CXXFLAGS="-march=armv6" go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_arm6 github.com/flynaj/kcptun/server
+if $UPX; then upx -9 client_linux_arm6 server_linux_arm6;fi
+tar -zcf kcptun-linux-arm6-$VERSION.tar.gz client_linux_arm6 server_linux_arm6
+$sum kcptun-linux-arm6-$VERSION.tar.gz
+
+# ARM-7
+ARMS=(7)
+#CC=arm-linux-gnueabihf-gcc-5 GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 CGO_CFLAGS="-march=armv7-a" CGO_CXXFLAGS="-march=armv7-a" go install std
+CC=arm-linux-gnueabihf-gcc-5 CXX=arm-linux-gnueabihf-g++-5 GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 CGO_CFLAGS="-march=armv7-a -fPIC" CGO_CXXFLAGS="-march=armv7-a -fPIC" go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_arm7  github.com/flynaj/kcptun/client
+CC=arm-linux-gnueabihf-gcc-5 CXX=arm-linux-gnueabihf-g++-5 GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 CGO_CFLAGS="-march=armv7-a -fPIC" CGO_CXXFLAGS="-march=armv7-a -fPIC" go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_arm7  github.com/flynaj/kcptun/server
+if $UPX; then upx -9 client_linux_arm7 server_linux_arm7;fi
+tar -zcf kcptun-linux-arm7-$VERSION.tar.gz client_linux_arm7 server_linux_arm7
+$sum kcptun-linux-arm7-$VERSION.tar.gz
+
+# ARM64
+CC=aarch64-linux-gnu-gcc-5 CXX=aarch64-linux-gnu-g++-5 GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_arm64  github.com/flynaj/kcptun/client
+CC=aarch64-linux-gnu-gcc-5 CXX=aarch64-linux-gnu-g++-5 GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_arm64  github.com/flynaj/kcptun/server
+if $UPX; then upx -9 client_linux_arm64 server_linux_arm64*;fi
+tar -zcf kcptun-linux-arm64-$VERSION.tar.gz client_linux_arm64 server_linux_arm64
+$sum kcptun-linux-arm64-$VERSION.tar.gz
+
+#MIPS32LE
+CC=mipsel-linux-gnu-gcc-5 CXX=mipsel-linux-gnu-g++-5 GOOS=linux GOARCH=mipsle CGO_ENABLED=1 GOMIPS=softfloat go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_mipsle github.com/flynaj/kcptun/client
+CC=mipsel-linux-gnu-gcc-5 CXX=mipsel-linux-gnu-g++-5 GOOS=linux GOARCH=mipsle CGO_ENABLED=1 GOMIPS=softfloat go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_mipsle github.com/flynaj/kcptun/server
+
+#MIPS32
+CC=mips-linux-gnu-gcc-5 CXX=mips-linux-gnu-g++-5 GOOS=linux GOARCH=mips CGO_ENABLED=1 GOMIPS=softfloat go build -ldflags "$LDFLAGS_LINUX"  -o client_linux_mips github.com/flynaj/kcptun/client
+CC=mips-linux-gnu-gcc-5 CXX=mips-linux-gnu-g++-5 GOOS=linux GOARCH=mips CGO_ENABLED=1 GOMIPS=softfloat go build -ldflags "$LDFLAGS_LINUX"  -o server_linux_mips github.com/flynaj/kcptun/server
+
+if $UPX; then upx -9 client_linux_mips* server_linux_mips*;fi
+tar -zcf kcptun-linux-mipsle-$VERSION.tar.gz client_linux_mipsle server_linux_mipsle
+tar -zcf kcptun-linux-mips-$VERSION.tar.gz client_linux_mips server_linux_mips
+$sum kcptun-linux-mipsle-$VERSION.tar.gz
+$sum kcptun-linux-mips-$VERSION.tar.gz
+
+echo "=== Building Completed ==="