Gone are the days of using ad hoc global wiring structures in chips to communicate between blocks. Why you ask? Well, it all started with the rise of multi-core processors. Earlier, there were very few blocks in a chip that needed to communicate with each other so we could easily go-by by using something like a bus as shown in the picture below.
Buses are just a collection of wires. Whatever data is being put on the bus can be read by every other block connected to the bus. Whether to consume that data or not is decided by decoding the data that is in the bus.
Now, as the number of blocks in a chip increased (which mostly started with increase in number of cores), implementing buses became a problem. From the picture above, it is very evident that the buses are not scalable. Even if we ignore the latency and bandwidth limitations in a bus, it is not easy to route tons and tons of wire in a chip. Also, after multi core processors, came the rise of System-On-Chip (SoC). This time, a chip not only has multiple cores, but other Intellectual Properties (IPs) also. So, basically not only the amount of data that needs to be communicated has increased, but also the varieties of data have increased. We needed something that could communicate all these data effectively and efficiently.
Well, we were in luck because the designers and academia professors were already researching into something that could do exactly what we wanted. It was called On-Chip Interconnection Networks (OCIN) and On-Chip Networks (OCN), but now-a-days, Network-On-Chip (NoC) or Fabric is the pervasive one. The paper in [1] gives an idea of how it all started (not the first paper in this topic, but it is a very relevant one). It was published in 2001, and I recommend all to give it a read. It is a rather interesting piece.
Now the network is not just a set of wires, it has a brain of its own. It has routers, buffers, adapters, etc. which can decide how and where to send the messages. The design of a NoC can be broken down into several building blocks, which are explained in brief below:
Topology – A NoC has channels (a single logical connection between routers) and nodes (routers). Topology determines the physical layout of these things in a chip, and it should be such that the possibility of deadlock is avoided to a greater extent. For example, an H-tree is a topology for distributing clocks evenly.
Routing – The routing algorithm determines how the message will be delivered to its destination in the network. The algorithm’s ability to balance traffic has a direct impact on the throughput of the network.
Flow Control – A network will have storage elements like buffers and FIFOs, and they will not always have space. A packet waiting to go to its destination might have to wait until these are free. Flow control mechanism is responsible for the allocation and deallocation of these elements. It is tightly coupled with routing strategy.
Router microarchitecture – A router can have components like input buffers, routing logic, crossbar, etc. All these things contribute to a delay in message getting transmitted to its destination, i.e. the latency. Due to this reason, a significant effort has been given to design the router microarchitecture to reduce the latency as much as possible.
Typical router microarchitecture [2]
With this, we finish the introduction of a NoC. In the next few articles, I will go into the details of a some of the NoC building blocks. Note that this article, or any other article in this topic is by no means a complete guide on NoC. I only aim to give the readers an idea of what it is like to be working as a NoC Designer. Please refer to the References below to learn about NoC in greater detail.
See you in the next article!
References:
[1] Dally, William & Towles, Brian. (2001). Route packets, not wires: On-chip interconnection networks. Proceedings - Design Automation Conference. 684 - 689. 10.1109/DAC.2001.156225.
[2] Networks-on-Chip: From Implementations to Programming Paradigms by Sheng Ma, Libo Huang, Mingche Lai, Wei Shi, Zhiying Wang
[3] On-Chip Networks, Second Edition by Natalie Enright Jerger, Tushar Krishna, Li-Shiuan Peh
[4] Principles and Practices of Interconnection Networks by William James Dally, Brian Patrick Towles
[5] Wu, Chang & Li, Yubai & Peng, Qicong & Chai, Song & Yang, Zhongming. (2009). Construction of a multidimensional plane network-on-chip architecture based on the hypercube structure. Progress in Natural Science - PROG NAT SCI. 19. 635-641. 10.1016/j.pnsc.2008.10.003.
[7] Computer Architecture - Lecture 20: Interconnects (Fall 2022) by Onur Mutlu (Link)
[8] Computer Architecture - Lecture 21: On-Chip Networks & Efficient Router Design (Fall 2022) by Onur Mutlu (Link)
[9] W. J. Dally, "Virtual-channel flow control," in IEEE Transactions on Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, March 1992, doi: 10.1109/71.127260.
[10] Next-Gen Computer Architecture: Till the End of Silicon by Smruti R. Sarangi
Comments