Dorova rečenica je prelepa: elegantna, nežna, uzvišena. Čovek barata jezikom kao retko ko, sve je tačno gde treba da bude, svaka reč pleše na neki način i čini da tekst odiše nekom lepotom koja se ne sreće često. Williams, F. C.; Kilburn, T.; Tootill, G. C. (Feb 1951), "Universal High-Speed Digital Computers: A Small-Scale Experimental Machine", Proc. IEE, 98 (61): 13–28, doi: 10.1049/pi-2.1951.0004, archived from the original on 2013-11-17. To continue the innovations and break the memory wall, we need to rethink the design of AI models. There are several issues here. First, the current methods for designing AI models are mostly ad-hoc, and/or involve very simple scaling rules. For instance, recent large Transformer models are mostly just a scaled version of almost the same base architecture proposed in the original BERT model [22]. Second, we need to design more data efficient methods for training AI models. Current NNs require a huge amount of training data and hundreds of thousands of iterations to learn, which is very inefficient. Some might note that it is also different from how human brains learn, which often only require very few examples per concept/class. Third, the current optimization and training methods need a lot of hyperparameter tuning (such as learning rate, momentum, etc.), which often results in hundreds of trial and error sweeps to find the right setting to train a model successfully. As such, the training cost reported in Figure 1 is only a lower bound of the actual overhead, and the true cost is typically much higher. Fourth, the prohibitive size of the SOTA NN models makes their deployment for inference very challenging. This is not just restricted to models such as GPT-3. In fact, deploying large recommendation systems (which are similar to Transformers but which have much larger embedding and very few MLP layers afterwards [23]) that are used by hyperscalar companies is a major challenge. Finally, the design of hardware accelerators has been mainly focused on increasing peak compute with relatively less attention on improving memory-bound workloads. This has made it difficult both to train large models, as well as to explore alternative models, such as Graph NNs which are often bandwidth-bound and cannot efficiently utilize current accelerators. When drawing time is up, ask the players to tape their scenes on the wall, forming a visual “memory cloud.”

Ahn J, Hong S, Yoo S, et al. A scalable processing-in-memory accelerator for parallel graph processing. In: Proceedings of 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), Portland, 2015. 105–117 Valavi H, Ramadge P J, Nestler E, et al. A 64-Tile 2.4-Mb in-memory-computing CNN accelerator employing charge-domain compute. IEEE J Solid-State Circ, 2019, 54: 1789–1799 Wu B, Wan A, Yue X, Keutzer K. Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. In2018 IEEE International Conference on Robotics and Automation (ICRA) 2018 May 21 (pp. 1887–1893). IEEE. Samsung Shows Industry's First 2-Gigabit DDR2 SDRAM". Samsung Semiconductor. Samsung. 20 September 2004 . Retrieved 25 June 2019.The term was coined in "Archived copy" (PDF). Archived (PDF) from the original on 2012-04-06 . Retrieved 2011-12-14. {{ cite web}}: CS1 maint: archived copy as title ( link). After both her parents die of cancer in the span of a few months, a young girl moves to Lithuania to live with her grandfather. Memory, grief,life. The results were stunning. Their new architectures could shorten sequence alignment time from 20 hours to less than a second. Center researchers also projected they could speed this up 100 times further in future evolutions of their processing redesigns.

a b c "Electronic Design". Electronic Design. Hayden Publishing Company. 41 (15–21). 1993. The first commercial synchronous DRAM, the Samsung 16-Mbit KM48SL2000, employs a single-bank architecture that lets system designers easily transition from asynchronous to synchronous systems.a b c d e "Late 1960s: Beginnings of MOS memory" (PDF). Semiconductor History Museum of Japan. 2019-01-23 . Retrieved 27 June 2019. Isobe, Mitsuo; Uchida, Yukimasa; Maeguchi, Kenji; Mochizuki, T.; Kimura, M.; Hatano, H.; Mizutani, Y.; Tango, H. (October 1981). "An 18 ns CMOS/SOS 4K static RAM". IEEE Journal of Solid-State Circuits. 16 (5): 460–465. Bibcode: 1981IJSSC..16..460I. doi: 10.1109/JSSC.1981.1051623. S2CID 12992820.