一 、Socket网络编程
Socket网络编程：socket通常也称作"套接字"，用于描述IP地址和端口，是一个通信链的句柄，应用程序通常通过"套接字"向网络发出请求或者应答网络请求。socket起源于Unix，而Unix/Linux基本哲学之一就是“一切皆文件”，对于文件用【打开】【读写】【关闭】模式来操作。socket就是该模式的一个实现，socket即是一种特殊的文件，一些socket函数就是对其进行的操作（读/写IO、打开、关闭）socket和file的区别：file模块是针对某个指定文件进行【打开】【读写】【关闭】socket模块是针对 服务器端 和 客户端Socket 进行【打开】【读写】【关闭】用法：sk = socket.socket(socket.AF_INET,socket.SOCK_STREAM,0)参数一：地址簇 socket.AF_INET IPv4（默认） socket.AF_INET6 IPv6 socket.AF_UNIX 只能够用于单一的Unix系统进程间通信参数二：类型 socket.SOCK_STREAM 流式socket , for TCP （默认） socket.SOCK_DGRAM 数据报式socket , for UDP socket.SOCK_RAW 原始套接字，普通的套接字无法处理ICMP、IGMP等网络报文，而SOCK_RAW可以；其次，SOCK_RAW也可以处理特殊的IPv4报文；此外，利用原始套接字，可以通过IP_HDRINCL套接字选项由用户构造IP头。 socket.SOCK_RDM 是一种可靠的UDP形式，即保证交付数据报但不保证顺序。SOCK_RAM用来提供对原始协议的低级访问，在需要执行某些特殊操作时使用，如发送ICMP报文。SOCK_RAM通常仅限于高级用户或管理员运行的程序使用。 socket.SOCK_SEQPACKET 可靠的连续数据包服务参数三：协议 0 （默认）与特定的地址家族相关的协议,如果是 0 ，则系统就会根据地址格式和套接类别,自动选择一个合适的协议1、第一个socket client端程序：向服务器端发送数据，接收服务器端发回来的数据
import socket client = socket.socket() # 声明socket类型，同时生成socket连接对象 client.connect((localhost, 8888)) client.send(bHello world!\n) # 要发送的数据 # 发送中文要进行编码 client.send(我要下载文件了,haha\n.encode("utf-8")) data = client.recv(1024) # 要收多少数据 print(recv:, data.decode()) # 接收有中文同样也要解码 client.close() import socket client = socket.socket() client.connect((localhost, 8888)) while True: msg = input(">>>:").strip() + "\n" client.send(msg.encode("utf-8")) data = client.recv(1024) print(recv:, data.decode()) client.close()2、第一个 socket server端程序：接收客户端发来的数据，并向客户端发送数据
import socket server = socket.socket() server.bind((localhost, 8888)) # 绑定要监听的端口 server.listen() # 监听，可以给一个整数参数表示是监听多少个客户端 print("我开始等消息了") conn, addr = server.accept() # 等消息进来, conn是对方请求连接的对象实例，addr是对方的地址 # conn 就是客户端连过来矶在服务器端为其生成的一个连接实例 print(conn, addr) print("消息进来了") #data = server.recv(1024) # 接收数据包大小 data = conn.recv(1024) # 要使用conn print(recv:, data.decode()) # 接收中文要解码 #server.send(data.upper()) conn.send(data.upper()) # 要使用conn server.close() #关闭 import socket server = socket.socket() # 创建实例server server.bind((localhost, 8888)) # 绑定端口 server.listen() # 监听端口 print("我开始等消息了") conn, addr = server.accept()# 将接收到的连接实例和地址赋值给两个变量 print(conn, addr) print("消息来了") while True: data = conn.recv() print(recv:, data.decode()) if not data: print("client has lost...") break conn.send(data.upper()) server.close() """ # /bin/env python # -*-coding:utf-8-*- # server import socket server = socket.socket() server.bind((localhost, 8888)) server.listen() print("我开始等消息了") conn, addr = server.accept() print(conn, addr) print("有消息来了") while True: data = conn.recv(0) print("recv: ", data) if not data: print("client has lost...") break conn.send(data.upper() """二、socket协议介绍
所有网络协议本质是对数据的收发tcp/ipsend recvudpfamily address(地址簇)AF.INET IPv4AF.INET6AF.UNIX localsocket protocol typesock.SOCK_STREAMtcp/ipsock.SOCK_DGRAM udp三 、socket服务器端和客户端服务建立流程
服务端建立socket流程：1、先声明实例：server = socket.socket(AF.INET, socket.SOCK_STREAM)2、绑定IP地址和端口server.bind((localhost, 8888))3、准备监听server.listen()4、连接客户端while True: # 断开之后，重新建立一个连接conn, addr = server.accept()# 阻塞# conn 是为每个客户端连接创建的实例，addr是客户端的IP地址while True:print("new conn", addr)data = conn.recv(1024)# 跟客户端通信的是conn，官方建议最大8192，recv默认是阻塞的if not data:break # 客户端已断开，conn.recv收到就是空数据print(data) # 可以打印数据conn.send(data.upper()) # 可以跟客户交互，发送回客户端# 客户端已断开， conn.recv收到的就是空数据，# 发送数据步骤：# 发送数据大小：conn.send(len(data))# 响应客户端：conn.recv(1024),在linux上解决发送数据粘包的问题# 正式发送数据：conn.send(data)客户端建立socket流程：1、实例化socketclient = socket.socket()#创建实例2、建立连接client.connect(serverip, 8888)3、发送数据、接收数据client.send(data)client.send(data)client.recv(data2)socket 粘包，注意解决粘包的思路1、第二个socket server端程序，接收客户端指令并执行，将执行结果发送给客户端
import socket, os, time server = socket.socket() #server.bind((localhost, 9999)) server.bind((0.0.0.0, 9999)) server.listen() while True: conn, addr = server.accept() print("New conn:", addr) while True: print("等待新指令") data = conn.recv(1024) if not data: print("客户端已断开") break print("执行指令:", data) cmd_res = os.popen(data.decode()).read() # 接受字符串，执行结果也是字符串 print("before send", len(cmd_res)) if len(cmd_res) == 0: cmd_res = "cmd has no output..." conn.send(str(len(cmd_res.encode())).encode("utf-8")) #time.sleep(0.5) # 睡0.5秒让缓冲区超时发送数据,解决粘包问题 # 另外一种解决思路是让客户端回复响应，解决粘包的问题 client_ack = conn.recv(1024) # wait client to confirm print("ack from client:", client_ack.decode()) conn.send(cmd_res.encode("utf-8")) # 在Linux上运行，会有数据“粘包”的情况，即两条send命令会粘在一起 print("send done") server.close()2、第二个socket client端程序
import socket client = socket.socket() # 创建实例 #client.connect((192.168.0.50, 9999)) # 连接linux client.connect((localhost, 9999)) while True: cmd = input(">>:").strip() # 输入指令 if len(cmd) == 0: continue # 输入为空，继续输入 client.send(cmd.encode("utf-8")) # 发送数据的时候一定要先编码 cmd_res_size = client.recv(1024) # 接收数据，首先接收数据大小 print("命令结果大小：", cmd_res_size) client.send("准备好接收了，loser可以发了".encode("utf-8")) received_size = 0 received_data = b while received_size < int(cmd_res_size.decode()): data = client.recv(1024) received_size += len(data) # 每次收到的有可能小于1024，所以必须使用len判断 #print(data.decode()) received_data += data else: print("cmd res receive done...", received_size) print(received_data.decode()) #print(cmd_res.decode()) client.close()3、第三个socket server服务端程序，实现FTP文件传送功能，解决粘包问题，linux上完全能正常运行。
import socket, os import hashlib server = socket.socket() server.bind((0.0.0.0, 9999)) server.listen() while True: conn, addr = server.accept() print("新连接:", addr) while True: print("waiting new command") data = conn.recv(1024) if not data: print("客户端已断开") break print("执行指令:", data) cmd,filename = data.decode().split() print(filename) if os.path.isfile(filename): f = open(filename, "rb") m = hashlib.md5() file_size = os.stat(filename).st_size conn.send( str(file_size).encode() )# send file size conn.recv(1024)# wait for ack,解决粘包问题 for line in f: m.update(line) conn.send(line) print("file md5", m.hexdigest()) f.close() conn.send(m.hexdigest().encode())# send md5 print("send done") server.close()4、第三个socket client端程序，FTP文件接收功能
import socket client = socket.socket()# 创建实例 client.connect((192.168.0.100, 9999)) #client.connect((localhost, 9999)) while True: cmd = input(">>:").strip() if len(cmd) == 0: continue if cmd.startswith("get"): client.send(cmd.encode()) server_response = client.recv(1024) print("server response:", server_response) client.send(b"ready to recv file") file_total_size = int(server_response.decode()) received_size = 0 filename = cmd.split()[1] f = open(filename, "wb") while received_size < file_total_size: data = client.recv(1024) received_size += len(data) f.write(data) print(file_total_size, received_size) else: print("file recv done") f.close() client.close()5、第四个socket client端程序，ftp客户端，使用方法：运行服务器和客户端程序后，在客户端输入：get filename，从服务器端下载filename的文件，对应的服务器端代码是第三个socket server程序
import socket,hashlib client = socket.socket()# 创建实例 #client.connect((192.168.0.100, 9999)) client.connect((localhost, 9999)) while True: cmd = input(">>:").strip() if len(cmd) == 0: continue if cmd.startswith("get"): client.send(cmd.encode()) server_response = client.recv(1024) print("server response:", server_response) client.send(b"ready to recv file") file_total_size = int(server_response.decode()) received_size = 0 filename = cmd.split()[1] + ".bak" f = open(filename, "wb") m = hashlib.md5() while received_size < file_total_size: if file_total_size - received_size > 1024:#要收不止一次 size = 1024 else: # 最后一次，剩多少收多少 size = file_total_size - received_size print("last receive:",size) data = client.recv(size) received_size += len(data) m.update(data) f.write(data) #print(file_total_size, received_size) else: new_file_md5 = m.hexdigest() print("file recv done", file_total_size, received_size) f.close() server_file_md5 = client.recv(1024) print("Server file md5:", server_file_md5.decode("utf-8")) print("client file md5:", new_file_md5) client.close()四、socketserver实现多线程并发连接
socket server:实现多用户发送数据First, you must create a request handler处理类 class by subclassing the BaseRequestHandler class and overriding覆盖 its handle() method; this method will process incoming requests. 你必须自己创建一个请求处理类，并且这个类要继承BaseRequestHandler,并且还有重写父亲类里的handle()Second, you must instantiate实例化 one of the server classes, passing it the server’s address and the request handler class.你必须实例化TCPServer ，并且传递server ip 和 你上面创建的请求处理类 给这个TCPServerThen call the handle_request() or serve_forever() method of the server object to process one or many requests.server.handle_request() #只处理一个请求server.serve_forever() #处理多个请求，永远执行Finally, call server_close() to close the socket.让你的socketserver并发起来， 必须选择使用以下一个多并发的类class socketserver.ForkingTCPServerclass socketserver.ForkingUDPServerclass socketserver.ThreadingTCPServerclass socketserver.ThreadingUDPServer1、第五个socket server程序：实现多线程并发连接，客户端程序使用第一个socket client程序实现
import socketserver class MyTCPHandler(socketserver.BaseRequestHandler): def handle(self): while True: try: self.data = self.request.recv(1024).strip() print("{} wrote:".format(self.client_address[0])) print(self.data) self.request.send(self.data.upper()) except (ConnectionAbortedError,ConnectionAbortedError) as e: print(error occurred,e) break if __name__== "__main__": HOST, PORT = "localhost", 8888 # Create the server, binding to localhost on port 9999 server = socketserver.TCPServer((HOST, PORT), MyTCPHandler) # 要使用多线程功能，就用ThreadingTCPServer # 多线程和多继承都是同时干多件事，多继承是 ForkingTCPServer #server = socketserver.ThreadingTCPServer((HOST, PORT), MyTCPHandler) # 在windows下不能使用ForkingTCPServer功能 server = socketserver.ForkingTCPServer((HOST, PORT), MyTCPHandler) server.serve_forever()2、socketserver介绍
class socketserver.BaseServer(server_address, RequestHandlerClass) 主要有以下方法class socketserver.BaseServer(server_address, RequestHandlerClass)This is the superclass of all Server objects in the module. It defines the interface, given below, but does not implement most of the methods, which is done in subclasses. The two parameters are stored in the respective server_address and RequestHandlerClass attributes.fileno()文件描述符Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to selectors, to allow monitoring multiple servers in the same process.handle_request()处理单个请求Process a single request. This function calls the following methods in order: get_request(), verify_request(), and process_request(). If the user-provided handle() method of the handler class raises an exception, the server’s handle_error() method will be called. If no request is received within timeout seconds, handle_timeout() will be called and handle_request() will return.serve_forever(poll_interval=0.5)Handle requests until an explicit(明确的) shutdown() request. Poll(检查) for shutdown every poll_interval seconds.Ignores the timeout attribute. It also calls service_actions(),which may be used by a subclass or mixin to provide actions specific to a given service. For example, the ForkingMixIn class uses service_actions() to clean up zombie僵尸 child processes.Changed in version 3.3: Added service_actions call to the serve_forever method.service_actions()This is called in the serve_forever() loop. This method can be overridden by subclasses or mixin classes to perform actions specific to a given service, such as cleanup actions.New in version 3.3.shutdown()Tell the serve_forever() loop to stop and wait until it does.server_close()Clean up the server. May be overridden.address_familyThe family of protocols to which the server’s socket belongs. Common examples are socket.AF_INET and socket.AF_UNIX.RequestHandlerClassThe user-provided request handler class; an instance of this class is created for each request.server_addressThe address on which the server is listening. The format of addresses varies depending on the protocol family; see the documentation for the socket module for details. For Internet protocols, this is a tuple containing a string giving the address, and an integer port number: (127.0.0.1, 80), for example.socketThe socket object on which the server will listen for incoming requests.The server classes support the following class variables:allow_reuse_addressWhether the server will allow the reuse of an address. This defaults to False, and can be set in subclasses to change the policy.request_queue_sizeThe size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to request_queue_size requests. Once the queue is full, further requests from clients will get a “Connection denied” error. The default value is usually 5, but this can be overridden by subclasses.socket_typeThe type of socket used by the server; socket.SOCK_STREAM and socket.SOCK_DGRAM are two common values.timeoutTimeout duration, measured in seconds, or None if no timeout is desired. If handle_request() receives no incoming requests within the timeout period, the handle_timeout() method is called.There are various server methods that can be overridden by subclasses of base server classes like TCPServer; these methods aren’t useful to external users of the server object.finish_request()Actually processes the request by instantiating RequestHandlerClass and calling its handle() method.get_request()Must accept a request from the socket, and return a 2-tuple containing the new socket object to be used to communicate with the client, and the client’s address.handle_error(request, client_address)This function is called if the handle() method of a RequestHandlerClass instance raises an exception. The default action is to print the traceback to standard output and continue handling further requests.handle_timeout()This function is called when the timeout attribute has been set to a value other than None and the timeout period has passed with no requests being received. The default action for forking servers is to collect the status of any child processes that have exited, while in threading servers this method does nothing.process_request(request, client_address)Calls finish_request() to create an instance of the RequestHandlerClass. If desired, this function can create a new process or thread to handle the request; the ForkingMixIn and ThreadingMixIn classes do this.server_activate()Called by the server’s constructor to activate the server. The default behavior for a TCP server just invokes listen() on the server’s socket. May be overridden.server_bind()Called by the server’s constructor to bind the socket to the desired address. May be overridden.verify_request(request, client_address)Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.