Gene Amdahl was a computer engineer in the purest sense of the word, having architected the legendary System/360 at IBM and pioneered other mainframe technologies at a series of his own enterprises.
Amdahl was in the graduate physics program at the University of Wisconsin in 1950, and along with fellow graduate student Charlie Davidson proposed the construction of a digital computer. This was only a few years after the ENIAC had become operational, and only two years after the first computer which could store a program. Physics requires a lot of math that’s tedious to calculate by hand, so machines that could automate much of the calculation and produce reliable results had an immediate audience with university physics departments, but the few computers built up to that point were in the domains of the military, IBM, or the most elite technical colleges.
Amdahl and Davidson received approval to start building their WISC computer, and before long Amdahl found the construction of the computer to be more fun than programming physics equations into it. Amdahl defended his doctorate in 1952 on the subject of the WISC computer and its internal processes, and left for IBM to work on its mainframe systems. WISC was completed in 1954 and Amdahl later donated it to the Computer History Museum.
In 1967 Amdahl was in his second tenure at IBM, and the System/360 had been commercially available for two years, when he attended the AFIPS’s (later known as the ACM) Spring Joint Computer Conference. Amdahl’s research article in the conference proceedings do not include a single thesis that states what we now know as Amdahl’s Law, so while there is no transcript of the debate Amdahl participated in on “The best approach to large computing capability ” we might conclude that the concise law came from the debate itself. Amdahl was arguing for a focus on single-processor capability before pursuing parallel computing to solve problems faster, and pointed out that major parts of programs on the computer systems of the time could not be feasibly parallelized. More concisely:
The speedup from parallelization is limited by the fraction of work that cannot be parallelized.
This principle is quite general. Even though it was presented in the context of the tradeoff of a single-processor computer system versus a parallelized multiple computer system, you could optimize a part of a single process instead, and still have to consider the same limitation. So in a computer system, a generalized Amdahl’s Law could be applied anywhere with a formulation like the one on Wikipedia:
The overall performance improvement gained by optimizing a single part of a system is limited by the fraction of time that the improved part is actually used
In fact, it applies to any system, including human operated ones. Because parallelizing a task usually involves increasing overhead, the speed increase is likely to experience decreasing marginal returns that diverge away from the theoretical limit. At the most extreme, when the overhead is communication channels and the effort required to train up new staff, the speedup could even become negative! Of course, we know that as Brooks’ Law.