type
status
slug
date
summary
tags
category
password
icon
3.2多路复用与多路分解:
将由网络层提供的主机到主机的交互服务延伸到为运行在主机上的应用程序提供进程到进程的交付服务。
补充一个什么是因特网(书第一章p1): 根据具体构成定义: The Internet is a computer network that interconnects billions of computing devices throughout the world. Not too long ago, these computing devices were primarily traditional desktop computers, Linux workstations, and so-called servers that store and transmit information such as Web pages and e-mail messages. Increasingly, however, users connect to the Internet with smartphones and tablets. Indeed, the term computer network is beginning to sound a bit dated, given the many nontraditional devices that are being hooked up to the Internet. In Internet jargon, all of these devices are called hosts or end systems 因特网是一个世界范围的计算机网络,即一个互联了遍及全世界的数十亿计算设备的网络。这些计算设备最初是传统的桌面PC、Linux工作站以及所谓的服务器(他们用于存储和传输Web页面、电子邮件等设备)。用因特网术语来说,所有这些设备都称为主机或端系统。 根据服务描述来定义: as an infrastructure that provides services to applications. In addition to traditional applications such as e-mail and Web surfing, Internet applications include mobile smartphone and tablet applications, including Internet messaging, mapping with real-time road-traffic information, music streaming movie and television streaming, online social media, video conferencing, multi-person games, and location-based recommendation systems. The applications are said to be distributed applications, since they involve multiple end systems that exchange data with each other. Importantly, Internet applications run on end systems—they do not run in the packet switches in the network core. Although packet switches facilitate(促进) the exchange of data among end systems, they are not concerned with the application that is the source or sink of data. 为应用程序提供服务的基础设施。除了电子邮件和浏览器等传统应用外,因特网应用还包括移动智能手机和平板电脑应用,其中包括即时通信、实施道路交通信息提醒、电影和电视节目、社交媒体、视屏会议、多人游戏等等。因为这些应用涉及多个相互交换数据的端系统上,即它们并不运行在网络核心中的分组交换机上。尽管分组交换机能够加速端系统之间的数据交换,但它们并不关注作为数据的源或宿应用。
将运输层报文段中的数据交付到正确的套接字的工作称为多路分解(demultiplexing)。
在源主机从不同套接字中收集数据块,并为每个数据块封装上首部信息(这将在以后用于分解)从而生成报文段,然后将报文段传递到网络层,所有这些工作称为多路复用(multiplexing)。
UDP:一个UDP套接字是由一个二元组全面标识的,该二元组包含一个目的IP地址和一个目的端口号。因此,如果两个UDP报文段有不同的源IP地址和/或源端口号,但具有相同的目的IP地址和目的端口号,那么这两个报文段将通过相同的目的套接字被定向到相同的目的进程。
TCP:差别是TCP套接字是由一个四源组(源IP地址,源端口号,目的IP地址,目的端口号)来识别的。因此,当一个TCP报文段从网络到达一台主机时,该主机使用全部4个值来将报文段定向(分解)到对应的套接字。两个具有不同源IP地址或源端口号的到达TCP报文段将被重定向到两个不同的套接字,除非TCP报文段携带了初始建立连接的请求。
这里展开说一下TCP的过程:
TCP服务器应用程序有一个“欢迎套接字”,我们假定它在12000号端口上等待来自TCP客户的连接请求
TCP客户端发送一个TCP连接建立请求(目的端口为12000,TCP首部特定“连接建立位”置位的TCP报文段),这个报文段包含了客户端的源端口号(一般透明)
当运行服务器进程的计算机的主机的操作系统接收到具有目的端口12000的入连接请求报文段后,它就定位服务器进程,该进程正在端口号12000等待接受连接。该服务器进程就会创建一个新的套接字
该服务器的运输层还注意到连接请求报文段中的下列4个值:
- 该报文段中的源端口号;
- 源主机IP地址
- 该报文段中的目的端口号
- 自身的IP地址
新创建的连接套接字就用过这4个值来标识。所有后续到达的报文段,如果它们的源端口号、源主机IP地址、目的端口号和目的IP地址都与这4个值匹配,则被分解到这个套接字。
UDP,无连接运输:
一个简单的模型:
UDP从应用层(应用进程)得到数据,附加上用于多路复用/分解服务的源和目的端口号字段,以及两个其他的小字段,然后将形成的报文段交给网络层。
这里需要辨析一下UDP报文段内容和UDP套接字(socket)的区别,UDP报文段的内容是包含源端口号和目的端口号的(准确一点是报文段的首部内容包含),不包含源IP地址和目的IP地址(这部分是属于网络层而不是传输层),但是,接收方接收到UDP报文,需要定位到对应的进程的时候,是通过一个二元组(目的IP地址,目的端口号)来找到对应的UDP套接字从而找到对应的进程。而这里目的IP地址就不是从UDP报文段里面找到的,而是从网络层的数据报里面找到的。其实同样的TCP报文段也只是包含了源端口和目的端口,并没有包含源IP地址和目的IP地址,然后TCP的套接字是通过四元组(源端口、目的端口,源IP地址、目的IP地址)来定位的,从而找到对应的进程。多的源信息,就是因为TCP是面向连接的,所以需要知道对方,不同的来源会有不同的套接字(对应进程/线程)来处理。
UDP里面长度字段是首部加上数据, UDP报文段首部只有4个字段—源端口号、目的端口号、长度、校验和。
校验和的计算:这两篇文章不错:
核心是这个步骤:
至于原因,知乎的文章解释的很好:把进位和加法分开计算。
3.4 可靠数据传输的原理
rdt1.0
rdt的发送端只通过rdt_send(data)事件接受来自高层的数据,产生一个包含该数据(经由
make_pkt(data)操作),并将分组发送到信道中(通过udt_send(packet)操作)。实际上,rdt_send(data)事件是由较高层应用的过程调用产生的(例如,rdt_send())
在接收端,rdt通过rdt_rcv(packet)事件从底层信道接收一个分组,从分组中提出数据(经由
extract(packet,data)操作),并将数据上传给较高层(通过deliver_data(data)操作)。实际上,rdt_rcv(packet)事件是由较底层协议的过程调用产生的(例如,rdt_rcv())
这里假设了下层信道是完全可靠的,有了完全可靠的信道,接收端就不需要提供任何反馈信息给发送方,因为不必担心出现差错。
这是最简单的情况。
rdt2.0
这个时候考虑所有发送的分组按其发送的顺序被接收,但是某些比特可能受损。想想前面学到的校验和(checksum)是不是可以用来检测某些比特受损的情况。
学术的表达,我们采用了自动重传请求(Automatic Repeat reQuest, ARQ)协议.
它依靠另外三种协议来处理存在比特差错的情况:
- 差错检测:可以用校验和,反正就是要用额外的比特(除了待发送的初始数据比特以外的比特)从发送方发送到接收方;
- 接收方反馈:发送方要了解到接收方情况(这里就是分组是否被正确接收)的唯一途径就是让接收方提供明确的反馈信息。我们rdt2.0就采用让接收方向发送方会送一个ACK与NAK分组。
- 重传。接收方收到有差错的分组时,发送方将重传该分组文。
rdt2.0的发送端有两个状态。
- 在最左边的状态中,发送端协议正等待来自上层传下来的数据。当rdt_send(data)事件出现时,发送方将产生一个包含待发送数据的分组(sndpkt),带有校验和, (通过make_pkt(data, checksum)),然后经由udt_send(sndpkt)操作发送该分组。
- 在最右边的状态中,发送方协议等待来自接收方的ACK或NAK分组。如果收到一个NAK分组,该协议重传上一个分组并等待
rd2.1和rd2.2合在一起讲解
先看看前面2.0有什么缺陷:
原理阐释
最关键一点就是ACK或者NACK可能会受损,并且如果一个ACK或NAK分组受损,发送方无法知道接收方是否正确接收了上一块发送的数据。
但是我们这个版本的解决方法都是重传分组(收到NACK或者说收到受损分组),尽管有可能这个时候接收方已经接收到了完整数据,只是它的ACK没有完整的到达发送方,但是我们仍然可以让发送方重新发送数据,只是接收方需要采取某种机制来鉴别这个数据是重传的还是新的,由于我们假定没有分组丢失,并且顺序不会打乱,所以只需要鉴别当前和上一次总共只有两次发送数据的序号,我们将序号标记为0、1,来给分组标定序号,那么问题就变得简单:
- 如果接收方已经正确接收到数据,并且发送了ACK,但是接收到了发送方发来的分组的序号是上一次的序号,那就说明是重传的数据,在这个状态下,这个事件触发的操作就是让接收方只是重新发送ACK数据(接收方和发送方发送的所有数据都会带有校验和来检验是否受损),让接收方知道数据已经接收了。当然,如果这次接收方接收到了完整数据,那么就会知道接收方接收到了上次的数据,接收方的状态会前进,不会再重传,但是如果这次的ACK还是没有完整到达,同样的,发送方会重传,接收方还会执行重复的操作,如上。
- 然而接收方如果没有接收到完整数据,同样发送NACK,如果这个NACK没有完整到达,发送方的处理和ACK没有完整送达或者NACK完整送达所执行的操作是一样的,都是重传,所以这就让我们想到,NACK这个单独的信息好像是可以被优化一下,省略掉的,这就引进了2.2版本的优化,简单叙述一下:
- 不再发送NAK,而是对上次正确接收的分组发送一个ACK(这个ACK会包含分组的序号来说明是对上一次的ACK),发送方接收到对同一个分组的两个ACK[即接收冗余ACK(duplicate ACK)]后,就知道接收方没有正确接收到跟在被确认两次的分组后面的分组。这个操作就还需要发送方去检查接收到的ACK报文中被确认的分组序号[这可以通过在发送方FSM中,在isACK()中包含参数0或1来实现]
先看2.1:
2.2部分只是减少了NACK,在发送方和接收方微调了规则
发送端:
- 注意到wait for ACK1 相对于2.1,把NAK的地方改成了isACK(rcvpkt,0)这里的0就是上一次接收到的ACK的分组的序号,所以这里产生冗余ACK,说明接收端没有收到,需要重新发送。 而在接收到正确数据的情况,判断isACK(rcvpkt,1),是否是接收到刚发送的数据的ACK返回信息。
- 再看wait for ACK0 同样的,相对于2.1,也是改成了isACK(rcvpkt,1)这里。而对于正确信息,也是相应改为了isACK(rcvpkt,0)。
接收端:
发送ACK的时候,对于接收到受损数据,或者是重传,但是已经接收过的数据,都是发送ack附带上上一分组的组号,拆开两个情况来看,接收到受损数据,那么原本应该是接收到组号0的话,那没收到,发送的组号是1,就会让发送方重新发送(因为ACK冗余),而如果是重传,原本已经接收到了组号0,然而发送方没有效接收到确认,仍然停留在上一组分组的发送等待确认环节,发送上一组分组号,也就是0(因为当前已经是在等待1号分组了),那么发送方就会得到确认,转到发送下一组的流程。
rdt3.0
我们开始考虑丢包的情况。
how to detect packet loss and what to do when packet loss occurs.
使用checksum,sequence numbers,ACK packets,and retransmission 能够解决后一个问题。但是我们需要一个新的方法解决第一个问题:检测丢包
这里有个习题问接收方的FSM,我认为和2.2的接收方没有区别:
提炼重点:
只是对发送方进行调整,在wait for ACK的两个状态里面的每个重传这里修改到由超时机制触发,而不是之前检测到包错误,或者是包序号不对,并且在wait for call 的两个状态上,新加入rdt_rcv事件,因为可能有包是晚到的,这样就在发送端进行抛弃处理,不接受,无操作。
3.4.2性能提升—Pipelined Reliable Data Transfer Protocols
Problem: rdt3.0 is a stop-and-wait protocol.
为啥利用率低? 就是我们数据在信号线里面传输的时间可能很长,但是你发送到信道的速度一般是很快的,一个例子就是你美国东海岸到西海岸,信息传输一个来回要30ms,但是你发送方发到数据信道(link)上的时间只是需要短短的8微秒,那这个浪费是惊人的,因为剩下的30ms时间,你发送方只能等待接收方返回,而不能再没有接收到接收方数据的情况下发送新的数据。
于是乎,我们想到一个简单的改进方式:不以停等方式运行,允许发送方发送多个分组而无需等待确认。这样运行起来就像一个流水线一样,一条条信息从发送方发送出去,接连不断,而后,由接收方再发送一天天接收信息,这样的流水线技术对可靠数据传输协议可带来如下影响:
- The range of sequence numbers must be increased, since each in-transit packet (not counting retransmissions) must have a unique sequence number and there may be multiple, in-transit, unacknowledged packets.
- The sender and receiver sides of the protocols may have to buffer more than one packet. Minimally, the sender will have to buffer packets that have been transmitted but not yet acknowledged. Buffering of correctly received packets may also be needed at the receiver, as discussed below.
- The range of sequence numbers needed and the buffering requirements will depend on the manner in which a data transfer protocol responds to lost, corrupted, and overly delayed packets. Two basic approaches toward pipelined error recovery can be identified: Go-Back-N and selective repeat.
3.4.3 Go-Back-N(GBN)
In a GO-BNacck-N(GBN) protocol, the sender is allowed to transmit multiple packets(when available) without waiting for an acknowledgment, but is constrained(被约束) to have no more than some maximum allowable number, N, of unacknowledged packets in the pipeline. We describe the GBN protocol in some detail in this section.
you can see base and nextseqnum where we define base to be the sequence number of the oldest unacknowledged packet and nextseqnum to be the smallest unused sequence number (that is, the sequence number of the next packet to be sent), then four intervals in the range of sequence numbers can be identified.
[0,base-1], Already ACK’d
[base,nextseqnum-1], sent, not yet ACK’d
[nextseqnum,base+N-1], Usable, not yet sent
[base,+++], Not usable
来看看它的FSM(扩展的FSM)
这里的start_timer就很重要:如果不新开start,那么在流式线式的批量发送下,第一条的ACK回来过后,不会触发base==nextseqnum,那么stop_timer不会执行,时间继续,同样的是第二条ACK,第三条…..,最终一定会触发timeout,到时又重新发一套从base到nextseqnum,虽然接收方能够处理冗余,但是这部分重发是完全没有必要的,并且会一直存在,重发也不止一次,那么我们每次接到ACK后重新启动归零定时器就很重要了,并且在最坏的情况下,几乎是一条一条,如停等方式发送的情况下,仍然是成功的,哪怕第一次ACK回来比第二次发送慢一点点,然后不会stop,那么也会start计时器,这个时候就是计时第二条packet的发送到接收过程了。而如果是是最好的情况,一条条连续发送,那么计时器的工作也是正常的。
上面这个图☝️很好的说明了GBN的工作过程:
该例是窗口长度N为4的GBN协议运行过程,因为窗口长度的限制,发送方发送分组0~3, 然后在继续发送之前,必须等待直到一个或多个分组被确认。当接收到每一个连续的ACK(例如ACK0 和 ACK1)时,该窗口便向前滑动,发送方便可以发送新的分组(分别是分组4和分组5)。在接收方,分组2丢失,因此分组3、4和5被发现是失序分组并被丢弃。
3.4.4 Selective Repeat(SR) 选择重传
问题提出:
在前面的GBN中,存在一个致命的问题,当我们窗口长度很大(如1000),带宽时延也很大(也就是一次packet传输的时间)的时候,单个分组的差错就能够引起GBN重传大量分组,然而正如上面的例子可见,pk3 pk4 pk5本不需要重传,因为已经被正确送达,但是仍然由于GBN的机制,需要重传,这就导致我们的流水线可能会被这些不必要重传的分组所充斥。
所以提出一个解决思路:
选择重传(Selective Repeat (SR)): avoid unnecessary retransmissions by having the sender retransmit only those packets that it suspects were received in error(that is, were lost or corrupted) at the receiver. This individual, as-needed, retransmission will require that the receiver individually acknowledge correctly received packets. A window size of N will again be used to limit the number of outstanding, unacknowledged packets in the pipeline. However, unlike GBN, the sender will have already received ACKs for some of the packets in the window.
Figure a shows the SR sender’s view of the sequence number space.
看看SR sender events and actions:
- Data received from above. When data is received from above, the SR sender checks the next available sequence number for the packet. If the sequence number is within the sender’s window, the data is packetized and sent; otherwise, it is either buffered or returned to the upper layer for later transmission, as in GBN.
- Timeout. Timers are again used to protect against lost packets. However, each packet must now have its own logical timer, since only a single packet will be transmitted on timeout. A single hardware timer can be used to mimic(模拟) the operation of multiple logical timers.
- ACK received. If an ACK is received, the SR sender marks that packet as having been received, provided it is in the window. If the packet’s sequence number is equal to send_base, the window base is moved forward to the unacknowledged packet with the smallest sequence number. If the window moves and there are untransmitted packets with sequence numbers that now fall within the window, these packets are transmitted.
而对于接收方,它的策略不再是之前GBN里面简单的接收正确顺序,正确内容的的分组,而是将确认一个正确接收的分组而不管其是否失序。失序的分组将被缓存知道所有丢失分组(即序号更小的分组)皆被收到为止,这时才可以将一批分组按序交付给上层。我们来详细列出采用的操作:
- Packet with sequence number in [rcv_base, rcv_base+N-1] correctly received. In this case, the received packet falls within the receiver’s window and a selective ACK packet is returned to the sender. If the packet was not previously received, it is buffered. If this packet has a sequence number equal to the base of the receive window(rcv_base in Figure b above), then this packet, and any previously buffered and consecutively numbered(beginning with rcv_base) packets are delivered to the upper layer. The receive window is then moved forward by the number of packets delivered to the upper layer. As an example, consider the Figure below. When a packet with a sequence number of rcv_base=2 is received, it and packets 3,4, and 5 can be delivered to the upper layer.
- Packet with sequence number in [rcv_base-N, rcv_base-1] is correctly received. In this case, an ACK must be generated, even though this is a packet that the receiver has previously acknowledged.
- Otherwise. Ignore the packet.
注意到接收方操作的第二步很重要,接收方重新确认(而不是忽略)已收到的那些序号小于当前窗口基序号的分组。You should convince yourself that this reacknowledgment is indeed needed. Given the sender and receiver sequence number spaces we give above. (Figer 3.23). for example, if there is no ACK for packet send_base propagating from the receiver to the sender, the sender will eventually retransmit packet send_base even though it is clear(to us, not the sender!) that the receiver has already received that packet. If the receiver were not to acknowledge this packet, the sender’s window would never move forward!
The sender and receiver will not always have an identical view of what has been received correctly and what has not. For SR protocols, this means that the sender and receiver windows will not always coincide(一致).
有一个关于序号大小范围和窗口大小范围关系的问题:
能看到,a(之前的ack都没有送达,pk0(对应第一个包的重发)先触发发送方超时,向接收方重传)和b(pk3未能发送,pk0(对应第5个包)发送成功)这两个情况,对于接收方而言,都是一样的情况,无法区分是重发还是新的包。
这就是因为我们的窗口选取太大,正确答案是窗口长度必须是小于等于序号空间大小的一半。why?
因为我们设窗口大小范围为N,序号大小范围为M。那么我们有两个情况,发出去丢了重发,和发新的,需要找到什么时候能够刚好区分开这两个情况:
第一次成功发出,窗口内N个都发出了并且接收到相应ACK,这个时候如果,N大于了M/2就会使得新的窗口一定包含了0序号,那么这样这个0序号是代表着第M+1个packet正确发出,然而这也和第一次没能接收到ACK的情况,进行重发第一个packet的情况重合,导致无法区分,而如果N小于等于M/2,那么哪怕第一次发送的packet全部送到,并且sender接收到了ACK,新的窗口的序号仍然保持在M/2~M之间,和第一次使用的序号是分隔开的,没有重合。
3.5 Connection-Oriented Transport: TCP(面向连接的运输)
underlying principles(基本原理)
TCP is said to be connection-oriented because before one application process can
begin to send data to another, the two processes must first “handshake” with each
other—that is, they must send some preliminary segments to each other to establish
the parameters of the ensuing data transfer.
The TCP “connection” is not an end-to-end TDM or FDM circuit as in a circuit-switched network.
Instead, the “connection” is a logical one, with common state residing(属于) only in the TCPs in the two communicating end systems.
the intermediate network elements do not maintain TCP connection state.
In fact, the intermediate routers are completely oblivious(未察觉到的) to TCP connections; they see datagrams, not connections.
A TCP connection provides a full-duplex service.
A TCP connection is also always point-to-point, that is, between a single sender and a single
receiver. So-called “multicasting”—the transfer of data from one sender to many receivers in a single send operation—is not possible with TCP.
这里浅提一下这个TCP连接的三次握手过程:
客户首先发送一个特殊的TCP报文段,服务器用另一个特殊的TCP报文段来响应,最后,客户再用第三个特殊报文段作为相应。前两个报文段不承载“有效载荷”,也就是不包含应用层数据;而第三个报文段可以承载有效载荷。由于这两台主机之间发送了3个报文段,所以这种连接建立过程就叫做三次握手(three-way handshake)
MSS的概念:
Note that the MSS is the maximum amount of application-layer
data in the segment, not the maximum size of the TCP segment including headers.
(This terminology is confusing, but we have to live with it, as it is well entrenched(根深蒂固的).)
注意到MSS是指在报文段里应用层数据的最大长度,而不是指包括首部的TCP报文段的最大长度。(该术语很容易混淆,但是我们不得不使用它,因为他已经根深蒂固了)
(TCP的典型值是1460=MTU(1500)-20(IP首部)-20(TCP首部)),UDP协议没有MSS的概念,传输层的数据可能一股脑的交给网络层,所以数据可能会被分片而影响性能:下面👇这个文章对于UDP说法有误,看后面一个CSDN的
所以TCP连接的组成包括:一台主机上的缓存、变量和与进程连接的套接字,以及另一台主机上的另一组缓存、变量和与进程连接的套接字。而在这两台主机之间的网络元素(路由器、交换机和中继器)中,没有为该连接分配任何缓存和变量。( no buffers or variables are allocated to the connection in the network elements(routers, switches, and repeaters)between the hosts.)
3.5.2 TCP Segment Structure
研究一下TCP报文段的结构。
一般是20字节(Options字段为空)
When TCP sends a large file, such as an image as part of a Web page, it typically breaks the file into chunks of size MSS(except for the last chunk, which will often be less than the MSS).
这里只需要一个大概即可,后面部分会详细阐释。
Source 和 Dest port (16 bit 0-65535)和UDP的一样,用来多路复用/多路分解→ 来自/送到 上层用的数据。同样还有检验和字段(16 bit)也是和UDP中的作用一致的。
下面说说不一样的:
The 32-bit sequence number field and the 32-bit acknowledgment number
field are used by the TCP sender and receiver in implementing a reliable data
transfer service, as discussed below.
The 16-bit receive window field is used for flow control. We will see shortly that
it is used to indicate the number of bytes that a receiver is willing to accept.
The 4-bit header length field specifies the length of the TCP header in 32-bit
words. The TCP header can be of variable length due to the TCP options field.
(Typically, the options field is empty, so that the length of the typical TCP header
is 20 bytes.)
The optional and variable-length options field is used when a sender and receiver
negotiate the maximum segment size (MSS) or as a window scaling factor for use
in high-speed networks. A time-stamping option is also defined. See RFC 854
and RFC 1323 for additional details.
The flag field contains 6 bits. The ACK bit is used to indicate that the value
carried in the acknowledgment field is valid; that is, the segment contains an
acknowledgment for a segment that has been successfully received. The RST、SYN and FIN
bits are used for connection setup and teardown, as we will discuss
at the end of this section. The CWR and ECE bits are used in explicit congestion
notification, as discussed in Section 3.7.2. Setting the PSH bit indicates that the
receiver should pass the data to the upper layer immediately. Finally, the URG bit is used to indicate that there is data in this segment that the sending-side upper-layer entity has marked as “urgent.”
The location of the last byte of this urgent data is indicated by the
16-bit urgent data pointer field. TCP must inform the receiving-side upper-layer entity when urgent data exists and pass it a pointer to the end of the urgent data. (In practice, the PSH, URG, and the urgent data pointer are not used. However, we mention these fields for completeness.)
(这里中文版翻译有误,应该是当有紧急数据存在时,TCP必须通知接收端的上层实体,并向其传递一个指向紧急数据结束的指针。)
Sequence Numbers and Acknowledgment Numbers
Sequence Numbers
The sequence number for a segment is therefore the byte-stream number of the first byte in the
segment.
The first segment gets assigned sequence number 0, the second segment gets assigned sequence number 1,000, the third segment gets assigned sequence number 2,000, and so on.
Acknowledgment numbers
The acknowledgment number that Host A puts in its segment is the sequence number of the next byte Host A is expecting from Host B
来看看例子吧
A telnet example :
suppose the starting sequence numbers are 42 and 79 for the client and server.
Recall that the acknowledgment number is the sequence number of the next byte of data that the host is waiting for. After the TCP connection is established but before any data is sent, the client is waiting for byte 79 and the server is waiting for byte 42.
唯一要以的是第三次返回:Seq=43,ACK=80这个时候,并没有data部分,但是Seq number仍然存在,因为TCP有一个Sequence number field,所以这个段必须得有一个sequence number,这里就填入发送成功后的下一个序号。
3.5.3 Round-Trip Time Estimation and Timeout
相比于之前已经探讨过的rdt3.0的方法,还有一些小问题,其中主要的是:超时间隔长度的设置。
显然 Timeout should be larger than the connection’s round-trip time(RTT), that is, the time from when a segment is sent until it is acknowledged. Otherwise, unnecessary retransmissions would be sent.
但是到底应该多大? 刚开始如何估计往返时间呢? 是否应该为所有未确认的报文段各设一个定时器?我们接下来讨论:
Estimating the Round-Trip Time
The sample RTT, denoted SampleRTT, for a segment is the amount of time between
when the segment is sent (that is, passed to IP) and when an acknowledgment for
the segment is received.
大多数TCP的实现仅在某个时刻做一次SampleRTT测量,而不是为每个发送的报文段测量一个SampleRTT。在任意时刻,仅为一个已发送的但目前尚未被确认的报文段测量SampleRTT,从而产生一个接近每个RTT的新SampleRTT值。另外,TCP绝不为已被重传的报文段计算SampleRTT;它仅为传输一次的报文段测量SampleRTT。(Instead of measuring a SampleRTT for every transmitted segment, most TCP implementations take only one SampleRTT measurement at a time. That is, at any point in time, the SampleRTT is being estimated for only one of the transmitted but currently unacknowledged segments, leading to a new value of SampleRTT approximately once every RTT.)
对于采样的RTT,我们会采取一种“平均”的方式,因为SampleRTT值会随着路由器的拥塞和端系统负载的变化而变化,所以我们会采取这样的手段:TCP维持一个SampleRTT均值(称为EstimatedRTT)。一旦获得一个新的SampleRTT时,TCP就会根据下列公式来更新EstimatedRTT:
The recommended value of α is α = 0.125 (that is, 1/8) [RFC 6298], in which case the formula above becomes:
this weighted average puts more weight on recent samples than on old samples. As the more recent samples better reflect the current congestion in the network. 为何说比旧的大?看起来旧的是0.875比0.125大,但是,哪怕是最近的旧的,也是乘上了一个0.125然后再乘了0.875那么肯定就小于0.875,所以说新的SampleRTT的权值是比旧的SampleRTT要大的。而这个衰减速度就是以的指数衰减的,所以这在统计学中称为指数加权移动平均(Exponential Weighted Moving Average EWMA)
除了估算RTT外,测量RTT的变化也是有价值的。
[RFC 6298] defines the RTT variation, DevRTT,
as an estimate of how much SampleRTT typically deviates from EstimatedRTT:
The recommended value of β is 0.25.
Note that DevRTT is an EWMA of the difference between SampleRTT and EstimatedRTT.
Setting and Managing the Retransmission Timeout Interval
现在,有了前面的EstimatedRTT和DevRTT,那我们该选择什么值来作为TCP的timeout interval呢?常用的,既保证比EstimatedRTT大(必须的),并且考虑了波动情况(波动大就等久点,波动小就等短点):
An initial TimeoutInterval value of 1 second is recommended [RFC 6298].
但是,这种情况下,当出现超时后,TimeoutInterval值将,以免即将被确认的后继报文段过早出现超时。然而,只要收到报文段并且更新了EstimatedRTT之后,就使用上述公式再次计算TimeoutInterval。
TCP also uses pipelining, allowing the sender to have multiple transmitted but yet-to-be-acknowledged segments outstanding(未完成的)at any given time.
3.5.4 Reliable Data Transfer
TCP creates a reliable data transfer service on top of IP’s unreliable best-effort service.
Recall that the Internet’s network-layer service (IP service) is unreliable. IP does
not guarantee datagram delivery(datagrams can overflow router buffers and never reach their destination), does not guarantee in-order delivery of datagrams(datagrams can arrive out of order), and does not guarantee the integrity(完整) of the data in the datagrams(bits in the datagram can get corrupted (flipped from 0 to 1 and vice versa)).
In our earlier development of reliable data transfer techniques, it was conceptually(概念上地) easiest to assume that an individual timer is associated with each transmitted but not yet acknowledged segment. But in this way, timer management can require considerable overhead(开支)
Thus, the recommended TCP timer management procedures [RFC 6298] use only a single retransmission timer, even if there are multiple transmitted but not yet acknowledged segments.
有三个主要事件:从上层应用程序接收数据;定时器超时;收到ACK。
第一个事件:TCP从应用程序接收数据,将数据封装(encapsulate)在一个报文段中,并把该报文段交给IP。每一个报文段都包含一个序号,这个序号就是该报文段第一个数据字节的字节流编号。如果定时器还没有为某谢谢其他报文段运行,则当报文段被传给IP时,TCP就启动定时器(将定时器想成是和最早的未被确认的报文段相关联)(think of the timer as being associated with the oldest unacknowledged segment)
第二个事件:TCP responds to the timeout event by retransmitting the segment that caused the timeout. TCP then restarts the timer.
第三个事件:On the occurrence of this event, TCP compares the ACK value y with its variable SendBase. The TCP state variable SendBase is the sequence number of the oldest unacknowledged byte. (Thus SendBase–1 is the sequence number of the last byte that is known to have been received correctly and in order at the receiver.)
As indicated earlier, TCP uses cumulative acknowledgments, so that y acknowledges the receipt of all bytes before byte number y. If y > SendBase, then the ACK is acknowledging one or more previously unacknowledged segments. Thus the sender updates its SendBase variable; it also
restarts the timer if there currently are any not-yet-acknowledged segments.
注意最后一句 It also restarts the timer if there currently are any not-yet-acknowledged segments.
为什么? 因为如果你不重置timer,那么timer当前记录的是之前最老的未确认报文段,然而现在最老的已经接收了,对于目前未确认的而言,它的发出时间肯定更晚,那么timer就会更容易超时,因为起始时间偏早,最坏的情况是目前的未确认报文是在ACK接收前的一瞬间才发出的,那么这个timer就多记录了一整个RTT,而重置它才是记录该未确认报文的发出时间,所以重置是很有必要的。
来几个例子就懂了:
Doubling the Timeout Interval(超时间隔加倍)
whenever the timeout event occurs, TCP retransmits the not-yet-acknowledged segment with the smallest sequence number. But each time TCP retransmits, it sets the next timeout interval to twice the previous value, rather than deriving it from the last EstimatedRTT and DevRTT.
This modification provides a limited form of congestion control. In times of congestion, if the sources continue to retransmit packets persistently, the congestion may get worse.
Fast Retransmit
One of the problems with timeout-triggered retransmissions is that the timeout period
can be relatively long.
When a segment is lost, this long timeout period forces the sender to delay resending the lost packet, thereby increasing the end-to-end delay.
To handle this: the sender can detect packet loss well before the timeout event occurs by noting so-called duplicateACKs. A duplicate ACK is an ACK that reacknowledges a segment for which the sender has already received an earlier acknowledgment.
To understand the sender’s response to a duplicate ACK, we must look at why the receiver sends a duplicate ACK in the first place.
主要就是这一条👇
当TCP接收方接收到一个序号大于下一个所期望的、按序的报文段时,它就检测到了数据流中的一个间隔(gap),这就是说明有报文段丢失。(larger than the next, expected, in-order sequence number, it detects a gap in the data stream)
This gap could be the result of lost or reordered segments within the network.
因为TCP没有使用显示的NAK,所以它会发送一个重复的ACK—reacknowledges the last in-order byte of data it has received.
(Note that Table 3.2 allows for the case that the receiver does not discard out-of-order segments.)
If the TCP sender receives three duplicate ACKs for the same data, it takes this as an indication
that the segment following the segment that has been ACKed three times has been lost.
In the case that three duplicate ACKs are received, the TCP sender performs a fast retransmit[RFC 6581], retransmitting the missing segment before that segment’s timer expires.
用下面的代码片段代替之前的ACK事件响应
GO-Back-N or Selective Repeat?(蛮有意思的,之前也想过这个问题)
Is TCP a GBN or an SR protocol?
的确TCP的ACK方式是:cumulative and correctly received but out-of-order segments are not individually ACKed by the receiver. 因此,TCP sender也只需要保存smallest sequence number of a transmitted but unacknowledged byte(SendBase) and the sequence number of the next byte to be sent(NextSeqNum). 这样看来,TCP就更像是一个GBN协议的实现。
但是果真如此吗?让我们看看不同吧,要知道绝大多数的TCP实现是多了一个缓存correctly received but out-of-order segments.
来看看这个情况:
当sender sends a sequence of segments 1,2,…, N, and all of the segments arrive in order without error at the receiver. Suppose that the acknowledgment for packet n<N gets lost, but the remaining N -1 acknowledgments arrive at the sender before their respective timeouts.
如果是GBN协议: retransmit not only the packet n, but also all of the subsequent packets n + 1,n + 2,…, N.
但是TCP:retransmit at most one segment, namely, segment n. If the acknowledgment for segment n + 1 arrived before the timeout for segment n, TCP would not even transmit segment n.
这种情况下,TCP的表现和GBN很不一样
有个修改意见就是: A proposed modification to TCP, the so-called selective acknowledgment [RFC 2018], allows a TCP receiver to acknowledge out-of-order segments selectively rather than just cumulatively acknowledging the last correctly received, in-order segment.
这样的话,TCP看起来就很像我们通常的SR协议。
所以总的来说,我们更应该把目前广泛使用的TCP实现方式(非RFC 2018的提议)看成是hybrid of GBN and SR protocols.
3.5.5 Flow Control 流量控制
TCP连接的两方都设有接收缓存。在TCP连接接收到正确、按序的字节后,就会把数据放在接收缓存里面。但是 If the application is relatively slow at reading the data in the buffer, the sender can very easily overflow the connection’s receive buffer by sending too much data too quickly.
因此我们提出了一个解决办法—flow-control service 来消除发送方使接收方溢出的可能性。
Flow control is thus a speed-matching service—matching the rate at which the sender is sending against the rate at which the receiving application is reading.
所以Flow-control service实际上就是一个速度匹配服务
这里需要有个辨析—congestion control:
TCP发送方因为IP网络的拥塞而被遏制被称为拥塞控制—congestion control(a TCP sender can also be throttled due to congestion within the IP network; this form of sender control is referred to as congestion control)
Even though the actions taken by flow and congestion control are similar (the throttling of the sender), they are obviously taken for very different reasons.
从例子出发:Host A is sending a large file to Host B over a TCP connection
先看Host B这边:
我们定义了:RcvBuffer是Host B的buffer大小
- LastByteRead: the number of the last byte in the data stream read from the buffer by the application process in B
- LastByteRcvd: the number of the last byte in the data stream that has arrived from the network and has been placed in the receive buffer at B
所以满足TCP不溢出,只需要满足以下不等式👇:
The receive window, denoted rwnd is set to the amount of spare room in the buffer:
再看Host A这边:
需要定义的变量是:LastByteSent和LastByteAcked.就是字面意思,最后发送的字节和最后被已知到ACK了的字节(注意累积式确认的方式)
由此我们就能得出还在发送中,没有被确认,放入Host B(接收端)缓冲区的字节数(unacknowledged data that A has sent into the connection):LastByteSent-LastByteAcked.
于是乎,我们只需要在发送端这边保证已交付给Ip的数据但是还没有被确认的数据的大小小于等于接收端接收窗口大小即可:
这里有个很细节的情况需要考虑:
想想当A给B发送了一个packet,此时更新了LastByteRcvd,这个时候如果恰好算得rwnd为0,返回给A,那么A就会认为B的缓冲区此时满了,所以不会再发送,而如果后续B没有需要返回给A报文,那么即使B清空了缓冲区,仍然不能让A知道,A就会被阻塞而不能再发送数据。
所以,为了解决这个问题,TCP协议有了如下的补充规定:
the TCP specification requires Host A to continue to send segments with one data byte when B’s receive window is zero. These segments will be acknowledged by the receiver. Eventually the buffer will begin to empty and the acknowledgments will contain a nonzero rwdn value.
有地方注意:A会发送一个data部分为1字节的包,这个是用来简化起见,这1字节并不会被接收端接收,接收端收到这个只会返回和之前一样的ACK以及最新的rwdn信息,并且如果此时rwdn还是0,那么仍然发送方A会再发送一个1字节data的包,seq部分是接收方发的ACK字段,然后一直询问是否rwdn空出来了。
Q: 这样是不是要求了接收方的buffer一定是发送方每次发送的data的整数倍?
A: 不完全对,通过动画里面设置File Size为16kb, Buffer Size为4kb,可以看到,最开始发送数据一次性填满了buffer(因为只要小于MSS就可以),然后B这边因为consumes一次只有2kb,当consume了2kb,返回rwnd为2kb时,发送方就会调整发送的包大小为2kb,而不是之前的4kb了。所以只要rwnd非0,发送方就可以发送packet,那么接收方就可以传回ACK以及相应的最新rwnd情况。
浅浅对比一下UDP,没有flow control:(这一段中文版翻译有误,实在是太烂了)
For a typical UDP implementation, UDP will append the segments in a finite-sized buffer that “precedes” the corresponding socket (that is, the door to the process).
也就是说,UDP会直接把接收报文加入到一个有限大小的缓冲区,这个缓冲区是在UDP socket之前,也就是在进程处理数据之前,所以如果进程没能及时从中读取数据,数据被冲掉就真的丢失了,接收方进程不会收到。(The process reads one entire segment at a time from the buffer. If the process does not read the segments fast enough from the buffer, the buffer will overflow and segments will get dropped.)
3.5.6 TCP Connection Management TCP连接管理
先从TCP连接建立开始说起:
- Step 1. The client-side TCP first sends a special TCP segment to the server-side TCP. This special segment contains no application-layer data. But one of the flag bits in the segment’s header, the SYN bit, is set to 1. In addition, the client randomly chooses an initial sequence number (client_isn) and puts this number in the sequence number field of the initial TCP SYN segment.
- Step 2. Once the IP datagram containing the TCP SYN segment arrives at the server host (assuming it does arrive!), the server extracts the TCP SYN segment from the datagram, allocates the TCP buffers and variables to the connection, and sends a connection-granted(允许连接) segment to the client TCP. This connection-granted segment also contains no application-layer data. But it contains three important pieces of information in the segment header:
First, the SYN bit is set to 1.
Second, the acknowledgment field of the TCP segment header is set to client_isn+1.
Finally, the server chooses its own initial sequence number (server_isn) and
puts this value in the sequence number field of the TCP segment header.
The connection-granted segment is referred to as a SYNACK segment.
- Step 3. Upon receiving the SYNACK segment, the client also allocates buffers and variables to the connection. The client host then sends the server yet another segment; this last segment acknowledges the server’s connection-granted segment (the client does so by putting the value server_isn+1 in the acknowledgment field of the TCP segment header). The SYN bit is set to zero, since the connection is established. This third stage of the three-way handshake may carry client-to-server data in the segment payload.(也就是第三阶段的segment可以捎带data)
再来看看连接的终止—All good things must come to an end.
连接结束,最重要的目的就是释放资源—buffers and variables
从例子出发吧:
the client decides to close the connection. client TCP sends a special TCP segment to the server process which has a flag bit in the segment’s header, the FIN bit, set to 1.
When the server receives this segment, it sends the client an acknowledgment segment in return.
Then, the server sends its own shutdown segment, which has the FIN bit set to 1.
Finally, the client acknowledges the server’s shutdown segment.
At this point, all the resources in the two hosts are now deallocated.
给一个客户端TCP会经历的状态图,我这里提出一些疑问,因为原文并没有很详细讲解实现:
Q1: 可以看到这里SYN_SENT状态是 第一次握手结束,等待第二次握手,而第二次握手收到发送方的ACK后,原文并没有说接收方发送了ACK,而是直接说进入了ESTABLISHED状态,这里我仔细想了想,第三次握手发送的ACK,如果是有数据发送的话,那么和正常的发送数据的segment内容是一模一样的,只有在接收方没有发送数据,这个时候需要接收方(发起连接请求方)主动发送一个确认给发送方,这样才会造成不同,所以确实大多数情况第三次握手是不特殊的。不过还是不理解为什么原文不说明白
Q2:这里TIME_WAIT状态,原文是说 The TIME_WAIT state lets the TCP client resend the final acknowledgment in case the ACK is lost. 那到底是什么情况下才会resend呢? 我的想法是,可能server端也有一个计时器,在发送FIN以后,如果一段时间没有接收到ACK,那么就重发一个FIN,而客户端就在这个TIME_WATI时间内等待,如果接收到了重复的FIN,说明server端没有接收到ACK,那么就重新发送。
下面是对应的服务器端的状态图:
对于前面的疑问,本书确实是没有仔细讲解,它原文说了:
We have not described what happens in certain pathological scenarios, for example, when both sides of a connection want to initiate or shut down at the same time. If you are interested in learning about this and other advanced issues concerning TCP, you are encouraged to see Stevens’ comprehensive book [Stevens 1994].
3.6 Principles of Congestion Control
Packet retransmission thus treats a symptom of network congestion but does not treat the cause of network congestion. To treat the cause of network congestion, mechanisms are needed to throttle(遏制) senders in the face of network congestion. (中文版翻译完全错了)
3.6.1 The Cause and the Costs of Congestion
先看问题,从问题出发,看看网络阻塞时会发生什么。
Scenario 1: Two Senders, a Router with Infinite Buffers
Let’s assume that the application in Host A is sending data into the connection
(for example, passing data to the transport-level protocol via a socket) at an average
rate of bytes/sec.
the rate at which Host A offers traffic to the router in this first scenario is thus bytes/sec.
Host B operates in a similar manner, and we assume for simplicity that it too is sending at a rate of bytes/sec. Packets from Hosts A and B pass through a router and over a shared outgoing link of capacity R.
Figure 3.44 plots the performance of Host A’s connection under this first scenario. The left graph plots the per-connection throughput(吞吐量) (number of bytes per second at the receiver) as a function of the connection-sending rate.
可以看到到R/2的时候,之后的接收方的吞吐量不再增加,而是保持在R/2这是因为这个上限会被两条连接之间的共享链路容量卡死。不能以超过R/2的稳定状态速率向接收方交付分组。
那可能觉得目标就是让我们的发送速率保持不超过并且足够接近R/2就是好的,因为充分利用了共享链路,但是这个时候的排队平均时延就会很大,为什么?这里需要想想:
如果说是理想情况,那么你这条路上一直就你们两个,并且保持稳定发挥,畅通无阻,那么发送速率
接近R/2,共享链路也能无积累的一直传输,但是我们有一个路由器,所以数据会由于路由器的存储转发传输机制(通常是这个,哪怕不是这个机制,路由器转发数据也会有传输时延,这是必然的),以及路由器的处理时延,这就导致了随着的增加,路由器缓冲区里面排队的数据会增长,这段排队时延增加了平均时延,甚至当无限接近R/2的时候,排队会无限长,那么平均时延将达到无限大。
也就是原文所说: Even in this (extremely) idealized scenario, we’ve already found one cost of a congested network—large queuing delays are experienced as the packet-arrival rate nears the link capacity.
Scenario 2: Two Senders and a Router with Finite Buffers
Let’s now slightly modify scenario 1 in the following two ways (see Figure 3.45).
第一个就是路由器buffer大小有限,所以会丢包了
第二个就是提供可靠传输,丢包/包损坏会被重传
更加精细化定义sending rate:
denote the rate at which the application sends original data into the socket by bytes/sec.
The rate at which the transport layer sends segments (containing original data and retransmitted data) into the network will be denoted bytes/sec.
is sometimes referred to as the offered load(供给载荷) to the network.
上面👆三个图像对应着三种重传方式,依次是:
- 完全能保证发送数据不超过buffer,则不会丢包,没有重传。虽然性能完美,但是现实不可能完美做到。
- 只会重传完全能够确认已经丢失的包,重传的都是真正丢了的。即使是这种情况下(有可能做到,通过设置足够长的超时重传时间),也会产生一个问题:发送方必须通过重传来修复因为缓冲区溢出而丢失的包。
- 会重传包,就是正常的实现方式(之前讨论的),这种情况下,几乎可以认为每个包都被重传了两次,尽管很容易做到,但是问题是致命的,会有不需要重传的包也会被重传(特别是网络阻塞时,时延很大,这种情况就很频繁发生),使得路由器会使用它的链路带宽去转发不需要转发的包。
Scenario 3: Four Senders, Routers with Finite Buffers, and Multihop Paths
这里 R2,可以看到如果特别大,在所有链路中(特别是A-C B-D)
因为B-D中R2是第一跳,而A-C中是第二条,所以很容易的,B-D的traffic at R2往往比A-C大很多,可能A-C就分不到R2的Buffer了,这样A-C的端到端吞吐量就急剧的下降到0,这张图就反映出这点:👇
这里还有个很关键的点:A-C的包是第二条才被丢弃掉的,那第一条路由器做的工作就完全被浪费掉了。要是不转发它,那么第一条路由器完全可以转发一个其他的包,只要不congestion到路由器R2就好了。
所以这样我们又能给出一个路由器congestion会带来的问题:
当一个分组沿一条路径被丢弃时,每个上游路由器用于转发该分组到丢弃该分组而使用的传输容量最终被浪费掉了。
when a packet is dropped along a path, the transmission capacity that was used at each of the upstream links to forward that packet to the point at which it is dropped ends up having been wasted.
3.6.2 Approaches to Congestion Control
分为两个方法:
- End-to-end congestion control. 不由网络层提供拥塞控制支持,而是由传输层,比如TCP,一般就是这种方法,去看报文段的丢失(超时或者3次冗余确认)
- Network-assisted congestion control. 这个就是由网络层辅助,也就是路由器向发送方提供关于网络中拥塞状态的显示反馈信息。关于这个就有两种反馈方式。Direct feedback 方式,一般就是采用choke packet的方式(将packet 直接中断,不再往后转发),就好像说我被噎住了,然后把东西吐了。另一种是相当于捎带,将阻塞状态放在一个field里面,捎带回发送方,会经历一次完成的round time trip的时间(network feedback via receiver)。
3.7 TCP Congestion Control
3.7.1 Classic TCP Congestion Control
主要思路就是让每个发送方根据阻塞情况减小流量(也就是相当于把发送到链路的流量作为检测到的网络阻塞情况的一个函数)(to have each sender limit the rate at which it sends traffic into its connection as a function of perceived network congestion.)
这样就会有三个自然而然的问题:
- How does a TCP sender limit the rate at which it sends traffic into its connection? TCP发送方怎么做到限制流量发送
- How does a TCP sender perceive that there is congestion on the path between itself and the destination? TCP发送方如何感知到路径上的阻塞
- What algorithm should the sender use to change its send rate as a function of perceived end-to-end congestion?应该使用什么算法/函数关系 来控制阻塞和发送速率的关系
我们依次来回答上述问题:
1、如何限速
很好想,之前我们flow control的时候不是加入了一个rwnd吗?给的公式是满足:
同样的,我们可以增加一个变量是表示阻塞程度的,记为cwnd(congestion window),限制改为:
2、如何感知阻塞
就是可以依靠检测丢失事件—timeout或者三次冗余ACK(快速重传机制);
顺带提一下,FSM和FSA(有限状态机)的关系:
就是有限自动机是没有输出的有限状态机。
3、限速的机制
这里给出最重要的FSM图:
其中,Fast recover状态是不必须的,没有它,则对于三次冗余ACK检验出的lost情况,也归在和time out一样的处理,走到Slow start状态。
没有Fast recover的是Tahoe的TCP早期版本,而有Fast recover的是较新版本的TCP Reno,给一个对比图:
初始ssthresh(slow start threshold)为1MSS,后来成倍增加,到8MSS的时候进入Congestion Avoidance,然后线性增加(每轮只会增加1MSS,而不是指数增加了),在12MSS的时候出现了三次冗余事件,由Tahoe的策略,则会返回到slow start 状态,cwnd置为1,ssthresh为原来cwnd减半,为6,而对于Reno,这里会进入Fast recover状态,ssthresh为原来cwnd减半,但是cwnd会变为ssthresh+前面三次的冗余ACK带来的3个MSS增加,这里就是9MSS,然后两者继续进行,我们的这个例子后面的流量阻塞情况一般,能够接到新的ACK,所以后续Reno也是很快进入congestion avoidance状态。
注意到图3.51同时也包含了where transmission of new segments or retransmitted segments can occur.
TCP Congestion Control: Retrospective 回顾
additive-increase, multiplicativedecrease (AIMD)
考虑到时间原因,后面的内容我搁置了,那部分内容也比较前沿,先把基础部分学了,再回来看,去看第四章了。
- 作者:liamY
- 链接:https://liamy.clovy.top/article/csnet/note/trans
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议,转载请注明出处。