推荐一款轻量级HTTP_HTTPS代理TinyProxy

众所周知,我们常用的  Nginx / Apache 都可以很方便的用来做为正向或反向代理服务器使用。但是它们都并不支持 HTTPS 的正向代理。

Nginx 做为正向代理不支持 HTTPS 的原因是因为 Nginx 没有实现 HTTP 1.1 Connect 方法。隧道的含义大约就是帮助无法完成 TLS 握手的代理服务器透传可以完成 TLS 握手的客户端请求,而不再解析流量中的内容。

关于 Connect 和隧道技术,可详见以下文章:

   •RFC 2817 ()

   •什么是 HTTP 隧道,怎么理解 HTTP 隧道呢? ()

今天我们来介绍一款同时支持 HTTP/HTTPS 的轻量级代理软件 TinyProxy,TinyProxy 支持以下功能特性:

•支持匿名模式。•支持 HTTPS,可以通过 CONNECT 请求来转发 HTTPS 连接。•远程监视:可远程查看日志和访问信息。•负载监视:可配置成当负载达到某个程度时,拒绝新的代理请求。•访问控制:可设置特定的 IP 地址或者 IP 段才可访问。•安全:不需要 root 权限。•轻量化:只需要极小的系统资源。•支持基于 URL 的过滤。•支持透明代理。•支持多级代理。

安装 TinyProxy

1. 通过软件包安装

TinyProxy 目前已支持大多数发行版通过软件包安装,下面介绍下比较常用的几个平台的安装方式。

•CentOS / RHEL

# 需要 EPEL 仓库$ yum install -y tinyproxy

•   Ubuntu / Debian

$ sudo apt-get -y install tinyproxy

如果你使用的是其它平台,更多的安装方式可直接参考官方文档:https://tinyproxy.github.io/

2. 通过源码安装

如果你使用的平台,官方还不支持通过软件包安装。你也可以通过源码进行编译安装。

$ git clone https://github.com/tinyproxy/tinyproxy.git$ cd tinyproxy$ ./autogen.sh$ ./configure$ make$ make install

配置 TinyProxy

TinyProxy 默认配置文件路径为 /etc/tinyproxy/tinyproxy.conf。如果你要自定义配置文件位置,可以在启动 TinyProxy 时 通过 -c 参数来指定。

$ cat /etc/tinyproxy/tinyproxy.conf#### tinyproxy.conf -- tinyproxy daemon configuration file#### This example tinyproxy.conf file contains example settings## with explanations in comments. For decriptions of all## parameters, see the tinproxy.conf(5) manual page.#### User/Group: This allows you to set the user and group that will be# used for tinyproxy after the initial binding to the port has been done# as the root user. Either the user or group name or the UID or GID# number may be used.#User nobodyGroup nobody## Port: Specify the port which tinyproxy will listen on.Please note# that should you choose to run on a port lower than 1024 you will need# to start tinyproxy using root.#Port 8888## Listen: If you have multiple interfaces this allows you to bind to# only one. If this is commented out, tinyproxy will bind to all# interfaces present.## Listen 192.168.0.1## Bind: This allows you to specify which interface will be used for# outgoing connections.This is useful for multi-homed machines where# you want all traffic to appear outgoing from one particular interface.##Bind 192.168.0.1## BindSame: If enabled, tinyproxy will bind the outgoing connection to the# ip address of the incoming connection.##BindSame yes## Timeout: The maximum number of seconds of inactivity a connection is# allowed to have before it is closed by tinyproxy.#Timeout 600## ErrorFile: Defines the HTML file to send when a given HTTP error# occurs.You will probably need to customize the location to your# particular install.The usual locations to check are:# /usr/local/share/tinyproxy# /usr/share/tinyproxy# /etc/tinyproxy##ErrorFile 404 "/usr/share/tinyproxy/404.html"#ErrorFile 400 "/usr/share/tinyproxy/400.html"#ErrorFile 503 "/usr/share/tinyproxy/503.html"#ErrorFile 403 "/usr/share/tinyproxy/403.html"#ErrorFile 408 "/usr/share/tinyproxy/408.html"## DefaultErrorFile: The HTML file that gets sent if there is no# HTML file defined with an ErrorFile keyword for the HTTP error# that has occured.#DefaultErrorFile "/usr/share/tinyproxy/default.html"## StatHost: This configures the host name or IP address that is treated# as the stat host: Whenever a request for this host is received,# Tinyproxy will return an internal statistics page instead of# forwarding the request to that host.The default value of StatHost is# tinyproxy.stats.##StatHost "tinyproxy.stats"### StatFile: The HTML file that gets sent when a request is made# for the stathost.If this file doesnt exist a basic page is# hardcoded in tinyproxy.#StatFile "/usr/share/tinyproxy/stats.html"## LogFile: Allows you to specify the location where information should# be logged to.If you would prefer to log to syslog, then disable this# and enable the Syslog directive.These directives are mutually# exclusive. If neither Syslog nor LogFile are specified, output goes# to stdout.#LogFile "/var/log/tinyproxy/tinyproxy.log"## Syslog: Tell tinyproxy to use syslog instead of a logfile.This# option must not be enabled if the Logfile directive is being used.# These two directives are mutually exclusive.##Syslog On## LogLevel: Warning## Set the logging level. Allowed settings are:#Critical(least verbose)#Error#Warning#Notice#Connect(to log connections without Infos noise)#Info(most verbose)## The LogLevel logs from the set level and above. For example, if the# LogLevel was set to Warning, then all log messages from Warning to# Critical would be output, but Notice and below would be suppressed.#LogLevel Info## PidFile: Write the PID of the main tinyproxy thread to this file so it# can be used for signalling purposes.# If not specified, no pidfile will be written.#PidFile "/var/run/tinyproxy/tinyproxy.pid"## XTinyproxy: Tell Tinyproxy to include the X-Tinyproxy header, which# contains the clients IP address.##XTinyproxy Yes## Upstream:## Turns on upstream proxy support.## The upstream rules allow you to selectively route upstream connections# based on the host/domain of the site being accessed.## Syntax: upstream type (user:pass@)ip:port ("domain")# Or: upstream none "domain"# The parts in parens are optional.# Possible types are http, socks4, socks5, none## For example:## connection to test domain goes through testproxy#upstream http testproxy:8008 ".test.domain.invalid"#upstream http testproxy:8008 ".our_testbed.example.com"#upstream http testproxy:8008 "192.168.128.0/255.255.254.0"### upstream proxy using basic authentication#upstream http user:pass@testproxy:8008 ".test.domain.invalid"### no upstream proxy for internal websites and unqualified hosts#upstream none ".internal.example.com"#upstream none "www.example.com"#upstream none "10.0.0.0/8"#upstream none "192.168.0.0/255.255.254.0"#upstream none "."### connection to these boxes go through their DMZ firewalls#upstream http cust1_firewall:8008 "testbed_for_cust1"#upstream http cust2_firewall:8008 "testbed_for_cust2"### default upstream is internet firewall#upstream http firewall.internal.example.com:80## You may also use SOCKS4/SOCKS5 upstream proxies:#upstream socks4 127.0.0.1:9050#upstream socks5 socksproxy:1080## The LAST matching rule wins the route decision.As you can see, you# can use a host, or a domain:#name matches host exactly#.namematches any host in domain "name"#.matches any host with no domain (in empty domain)#IP/bitsmatches network/mask#IP/maskmatches network/mask##Upstream http some.remote.proxy:port## MaxClients: This is the absolute highest number of threads which will# be created. In other words, only MaxClients number of clients can be# connected at the same time.#MaxClients 100## MinSpareServers/MaxSpareServers: These settings set the upper and# lower limit for the number of spare servers which should be available.## If the number of spare servers falls below MinSpareServers then new# server processes will be spawned.If the number of servers exceeds# MaxSpareServers then the extras will be killed off.#MinSpareServers 5MaxSpareServers 20## StartServers: The number of servers to start initially.#StartServers 10## MaxRequestsPerChild: The number of connections a thread will handle# before it is killed. In practise this should be set to 0, which# disables thread reaping. If you do notice problems with memory# leakage, then set this to something like 10000.#MaxRequestsPerChild 0## Allow: Customization of authorization controls. If there are any# access control keywords then the default action is to DENY. Otherwise,# the default action is ALLOW.## The order of the controls are important. All incoming connections are# tested against the controls based on order.#Allow 127.0.0.1# BasicAuth: HTTP "Basic Authentication" for accessing the proxy.# If there are any entries specified, access is only granted for authenticated# users.#BasicAuth user password## AddHeader: Adds the specified headers to outgoing HTTP requests that# Tinyproxy makes. Note that this option will not work for HTTPS# traffic, as Tinyproxy has no control over what headers are exchanged.##AddHeader "X-My-Header" "Powered by Tinyproxy"## ViaProxyName: The "Via" header is required by the HTTP RFC, but using# the real host name is a security concern.If the following directive# is enabled, the string supplied will be used as the host name in the# Via header; otherwise, the servers host name will be used.#ViaProxyName "tinyproxy"## DisableViaHeader: When this is set to yes, Tinyproxy does NOT add# the Via header to the requests. This virtually puts Tinyproxy into# stealth mode. Note that RFC 2616 requires proxies to set the Via# header, so by enabling this option, you break compliance.# Dont disable the Via header unless you know what you are doing...##DisableViaHeader Yes## Filter: This allows you to specify the location of the filter file.#Filter "/etc/tinyproxy/filter"## FilterURLs: Filter based on URLs rather than domains.##FilterURLs On## FilterExtended: Use POSIX Extended regular expressions rather than# basic.##FilterExtended On## FilterCaseSensitive: Use case sensitive regular expressions.##FilterCaseSensitive On## FilterDefaultDeny: Change the default policy of the filtering system.# If this directive is commented out, or is set to "No" then the default# policy is to allow everything which is not specifically denied by the# filter file.## However, by setting this directive to "Yes" the default policy becomes# to deny everything which is _not_ specifically allowed by the filter# file.##FilterDefaultDeny Yes## Anonymous: If an Anonymous keyword is present, then anonymous proxying# is enabled.The headers listed are allowed through, while all others# are denied. If no Anonymous keyword is present, then all headers are# allowed through.You must include quotes around the headers.## Most sites require cookies to be enabled for them to work correctly, so# you will need to allow Cookies through if you access those sites.##Anonymous "Host"#Anonymous "Authorization"#Anonymous "Cookie"## ConnectPort: This is a list of ports allowed by tinyproxy when the# CONNECT method is used.To disable the CONNECT method altogether, set# the value to 0.If no ConnectPort line is found, all ports are# allowed.## The following two ports are used by SSL.##ConnectPort 443#ConnectPort 563## Configure one or more ReversePath directives to enable reverse proxy# support. With reverse proxying its possible to make a number of# sites appear as if they were part of a single site.## If you uncomment the following two directives and run tinyproxy# on your own computer at port 8888, you can access Google using# :8888/google/ and Wired News using# :8888/wired/news/. Neither will actually work# until you uncomment ReverseMagic as they use absolute linking.##ReversePath "/google/"""#ReversePath "/wired/"""## When using tinyproxy as a reverse proxy, it is STRONGLY recommended# that the normal proxy is turned off by uncommenting the next directive.##ReverseOnly Yes## Use a cookie to track reverse proxy mappings. If you need to reverse# proxy sites which have absolute links you must uncomment this.##ReverseMagic Yes## The URL thats used to access this reverse proxy. The URL is used to# rewrite HTTP redirects so that they wont escape the proxy. If you# have a chain of reverse proxies, youll need to put the outermost# URL here (the address which the end user types into his/her browser).## If not set then no rewriting occurs.##ReverseBaseURL ":8888/"

下面我们来看下几个主要的配置参数:

•   User

指定运行 TinyProxy 的用户,默认为 nobody。

User nobody

•   Group

指定运行 TinyProxy 的用户组,默认为 nobody。

Group nobody

•   Listen

指定 TinyProxy 绑定的网卡接口,默认是绑定到所有可用的网卡接口的。

#Listen 192.168.0.1

如需绑定到指定网卡接口,只需去掉对应的注释并指定网卡对应 IP 地址即可。

Listen 192.168.1.100

•   Port

指定 TinyProxy 的监听端口, 默认为 8888。

Port 8888

•   Allow

指定可访问 TinyProxy 设备的 IP 或网段,默认仅允许本机访问。

Allow 127.0.0.1

如果你想允许所有人使用该代理,注释 Allow 选项即可。

# Allow 127.0.0.1

如果你想增加多个可访问的网段,可以用多个 Allow 选项同时定义不同网段即可。

# 添加多段 IP 地址Allow 10.10.6.0/24Allow 192.168.8.0/24Allow 172.16.1.13

•BindSame在多网卡的情况下,设置出口 IP 是否与入口 IP 相同。默认情况下是关闭的。

例如:服务器上存在 IP 1.2.3.4,当你请求该 IP 对应的 Tinyproxy 代理时,也通过 1.2.3.4 做为出口访问目标网站。

#BindSame yes

•   StartServers

指定 TinyProxy 初始启动的子进程数量, 默认是 10 个。

StartServers 10

•   MaxClients

设置最大客户端链接数,默认为 100。

MaxClients 100

•   Logfile

指定日志文件位置, 默认为 /var/log/tinyproxy/tinyproxy.log。

LogFile /var/log/tinyproxy/tinyproxy.log

•   Syslog

指定 TinyProxy 是否开启 Syslog 来记录日志,默认为关闭的。

#Syslog On

注:Logfile 和 Syslog 只能同时启用一个。如果两个都不启用的话 TinyProxy 会将日志直接输出到终端的标准输出。

•   PidFile

指定 Pid 文件位置, 默认为 /var/run/tinyproxy/tinyproxy.pid,在 PidFile 文件不存在时会运行失败。

PidFile "/var/run/tinyproxy/tinyproxy.pid"

•   DisableViaHeader

指定是否在 Header 中显示 Tinyproxy 相关信息,默认是关闭的。如果开启将不会在 Header 中显示 Tinyproxy 相关信息,相当于 Tinyproxy 是隐身模式。

#DisableViaHeader Yes

•   Filter

指定设置过滤内容文件的位置,默认为 /etc/tinyproxy/filter。

Filter "/etc/tinyproxy/filter"

•   FilterURLs

设置使用 URL 或是域名方式进行过滤,默认是基于 URL 方式过滤的。域名过滤只检查域名段,URL 过滤则检查整个 URL。

FilterURLs On

•   FilterExtended

设置使用 POSIX 基本或者扩展的正则表达式来匹配过滤规则,默认为使用基本的。

# FilterExtended On

•   FilterCaseSensitive

设置是否使用区分大小写的正则表达式,默认为不区分大小写。

#FilterCaseSensitive On

•   FilterDefaultDeny

设置默认过滤策略。如果将该指令注释掉或设为 No,过滤规则为禁止访问规则。该值默认为 Yes,过滤规则为只允许访问过滤文件中的地址。

FilterDefaultDeny Yes

过滤规则配置示例:

1. 在 /etc/tinyproxy/filter 文件中添加代理允许或拒绝的域名地址。

hi-linux.com

过滤文件中的域名地址也是支持正则表达式的。

\.google\.com$^hi-linux\.com$

2. 仅允许代理请求 hi-linux.com 的内容,配置如下:

Filter "/etc/tinyproxy/filter"FilterURLs OnFilterDefaultDeny Yes

3. 仅允许代理请求除 hi-linux.com 域名以外的内容,配置如下:

Filter "/etc/tinyproxy/filter"FilterURLs OnFilterDefaultDeny No

运行 TinyProxy

•   运行 TinyProxy 非常简单,使用官方提供的脚本即可。

# 启动 TinyProxy$ service tinyproxy start# 停止 TinyProxy$ service tinyproxy stop# 重启 TinyProxy$ service tinyproxy restart

•   如果服务器有启用防火墙,记得开放相应的 TinyProxy 端口

$ iptables -I INPUT -p tcp –dport 8888 -j ACCEPT

•   测试代理是否正常工作

$ curl --proxy 192.168.1.100:8888 -k

如果出现对应网页的源代码,则证明代理工作正常。

•   查看 TinyProxy 请求日志

$ tail -f /var/log/tinyproxy/tinyproxy.log

参考文档

今日思想

你的时间有限,所以不要为别人而活。不要被教条所限,不要活在别人的观念里。不要让别人的意见左右自己内心的声音。最重要的是,勇敢的去追随自己的心灵和直觉,只有自己的心灵和直觉才知道你自己的真实想法,其他一切都是次要。

—— 乔布斯

推荐阅读

lsof 命令使用指南

图解 Kubernetes 架构

谈谈监控那些事

抓包神器 TCPDUMP 使用教程

Linux 并行处理神器 GNU Parallel 简明教程